-- Logs begin at Thu 2022-08-18 10:32:25 UTC, end at Thu 2022-08-18 11:44:53 UTC. -- Aug 18 11:01:35.684595 dv-cinder-853450-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 18 11:01:35.692367 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 18 11:01:35 2022] *** Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: nodename: dv-cinder-853450-2-iscsi Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: machine: x86_64 Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: clock source: unix Aug 18 11:01:35.694355 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: pcre jit disabled Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: detected number of CPU cores: 4 Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: current working directory: / Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: detected binary path: /usr/bin/uwsgi-core Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: your processes number limit is 47680 Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: your memory page size is 4096 bytes Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: detected max file descriptor number: 2048 Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: lock engine: pthread robust mutexes Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: thunder lock: enabled Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 18 11:01:35.694804 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 18 11:01:35.735087 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: Python main interpreter initialized at 0x555ba64129c0 Aug 18 11:01:35.735087 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: python threads support enabled Aug 18 11:01:35.735087 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: your server socket listen backlog is limited to 100 connections Aug 18 11:01:35.735087 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: your mercy for graceful operations on workers is 90 seconds Aug 18 11:01:35.736015 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: mapped 403077 bytes (393 KB) for 2 cores Aug 18 11:01:35.736156 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: *** Operational MODE: preforking *** Aug 18 11:01:35.736351 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: *** uWSGI is running in multiple interpreter mode *** Aug 18 11:01:35.736351 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: spawned uWSGI master process (pid: 81610) Aug 18 11:01:35.736557 dv-cinder-853450-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 18 11:01:35.737768 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: spawned uWSGI worker 1 (pid: 81624, cores: 1) Aug 18 11:01:35.737768 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: spawned uWSGI worker 2 (pid: 81625, cores: 1) Aug 18 11:01:35.737768 dv-cinder-853450-2-iscsi devstack@keystone.service[81610]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 18 11:01:40.048931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 11:01:40.048931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Aug 18 11:01:40.049631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 11:01:40.049631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: from cryptography.utils import int_from_bytes Aug 18 11:01:40.050082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 11:01:40.050082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Aug 18 11:01:40.050211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 11:01:40.050211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: from cryptography.utils import int_from_bytes Aug 18 11:01:40.100476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 11:01:40.100673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 11:01:40.100763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 11:01:40.100851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] command line args: [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 11:01:40.100965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 11:01:40.101097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 11:01:40.101190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] command line args: [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 11:01:40.101263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ================================================================================ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 11:01:40.101413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 11:01:40.101499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.101596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ================================================================================ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 11:01:40.101748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_dir = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.101834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] admin_token = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.101937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_dir = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_source = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] debug = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_source = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] debug = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.102987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_config_append = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.103985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_config_append = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_dir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_dir = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.104958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.105868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106729 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.106952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.107950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] publish_errors = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] publish_errors = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.108938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] standard_threads = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.109959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] standard_threads = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_journal = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_json = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.110981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_journal = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_json = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_stderr = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_syslog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_stderr = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_syslog = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 11:01:40.111775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.111955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.112974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.113933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.114979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.115927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.116930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.117917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.118855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.119963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.120928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.121926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.122326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.122404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.122547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.122625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.122786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.122867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.123948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.124905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.125009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.125143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.125290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.125389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.125505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.125633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.126955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.127898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.128881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.129844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.130951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.131966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.132961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.133904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.134960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.135969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.136885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.external = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.external = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.137916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.password = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.token = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.password = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.token = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.138886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.139836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.140978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.141056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.141226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.141306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.142896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.143888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.144935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145651 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.145893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.146849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.147960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.148946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.149969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.150908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.151890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.152902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.153985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.154934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.155776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.156998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.157993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.158935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.159899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.160952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.161954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.162941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.163951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.164979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.165997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.166910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.167935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.168981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.169977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.170984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.171993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.172116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.172271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.172395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.172518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.172741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.172947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.173134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.173366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.173492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.173615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.173748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.173877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.174880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.175891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.176783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.177889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.178915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.179923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.180963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.181955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.182970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.183971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.184890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.185933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.186979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.187996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.188118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.188788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.188923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.189994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.190052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.190111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.190170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.190237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.190298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 11:01:40.190382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 11:01:40.190382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 11:01:40.218349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.218578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.218758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.218847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.218996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.219233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.227758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.228144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.257173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.257354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.257447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.257539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.311319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.311459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.359996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.360778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.360865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.361082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.521066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.521306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.521521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.521713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.521829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.521949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.522033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.522165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.522249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.522384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.522604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.522822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.544957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.545958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.546094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.546179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.546311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.546394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.546548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.546764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.557947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.558389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.571405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.571593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.571798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.571898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.572882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.573095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 11:01:40.589411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.589623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 11:01:40.589959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.590044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.590159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 11:01:40.590257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 11:01:40.590440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.590564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.590687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 11:01:40.590770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.590966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.591217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.591299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.591489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.591730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.591998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.601588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.601870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.602044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.602147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.602326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.602421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.602600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.602887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.609433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.609865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.612937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.613225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.613394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.613511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.613683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.613798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.613969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.614890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.615082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.615278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.615370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.615562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.615732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.615837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.616015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.616315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.638772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.639130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.640262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.640553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.640637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.640893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.645314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.645604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.645688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.645888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.645985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.646177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.652282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.652547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.652632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.652836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.656677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.657488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.657626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.657777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.657860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.658977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.659164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.659246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.659430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.677073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.677177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.677353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.677437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.681261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.681404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.684502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.684600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.684805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.684896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.689236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.689333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.692403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.692530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.695649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.695750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.695963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.696947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.697149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.697241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.713131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.713231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.713429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.713513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.713703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.713798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.713972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.714054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.721542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.721635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.723059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.723182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.723341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.723466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.726000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.726128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.726288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.726422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.726573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.726692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.726889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.727003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.727173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.727289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.739307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.739533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.739690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.739823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.740109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.740209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.740424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.740589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.740710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.740992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.741073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.741281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.741488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.741810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.757163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.757466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.760218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.760533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.763084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.763428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.764531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.764896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.766005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.766291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.766433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.766706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.766788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.767047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.773197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.773538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.774642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.774986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.777573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.777874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.777956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.778164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.778245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.778441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.778522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.778717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.786875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.787172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.787254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.787454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.787549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.787786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.794358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.794642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.794724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 11:01:40.794925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.795855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.796977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.797165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.797247 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.797437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.797527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.798254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.798380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.798508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.798591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.798788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 11:01:40.832509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555ba64129c0 pid: 81625 (default app) Aug 18 11:01:40.832698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555ba64129c0 pid: 81624 (default app) Aug 18 11:01:40.835833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2409c-1c9b-4244-9dd1-ec62620db548 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:40.836120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2409c-1c9b-4244-9dd1-ec62620db548 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:40.836422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2409c-1c9b-4244-9dd1-ec62620db548 None None] PATH_INFO: `/v3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:40.840539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Aug 18 11:01:40 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:01:47.441144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:47.441654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:47.441737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:47.446073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:01:47.451793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:47.452254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:47.452367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:47.529052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: INFO dbcounter [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] Registered counter for database keystone Aug 18 11:01:47.549316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG oslo_db.sqlalchemy.engines [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 11:01:47.555426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writer thread running {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 11:01:47.734800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81625) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 11:01:47.735177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81625) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 11:01:47.790165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:47.791767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:47.922313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 18 11:01:48.374982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:48.444526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: INFO dbcounter [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] Registered counter for database keystone Aug 18 11:01:48.466288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG oslo_db.sqlalchemy.engines [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 11:01:48.473362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writer thread running {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 11:01:48.592951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de397884-3151-4935-bec0-bfb984b1aeb6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:48.597940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:48.598563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:48.598836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:48.599099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:48.607049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:48.659554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING keystone.common.rbac_enforcer.enforcer [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 11:01:48.659893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25624051-b37b-4517-89e3-5b57e381f4c5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:48.669196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Aug 18 11:01:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:01:48.675134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:48.689550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6218-83ec-44f7-9d35-e57cd4e7be42 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:48.693919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:48.694998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:48.695277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:48.695545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] PATH_INFO: `/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:48.697095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] RBAC: Authorizing `identity:get_project(project_id=c8a7d63a241745a88c5287cb0d12f0c9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:48.750664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING keystone.common.rbac_enforcer.enforcer [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 11:01:48.751005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cc643-9fe9-4258-ab07-a19e8f9ae612 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:48.755469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Aug 18 11:01:48 2022] GET /identity/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:01:50.639799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684628-90e8-466a-a0d8-959bce3d4c3c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:50.640377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684628-90e8-466a-a0d8-959bce3d4c3c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:50.640377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684628-90e8-466a-a0d8-959bce3d4c3c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:50.644472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:01:50.649707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:50.650131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:50.650262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:50.670455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:50.671990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:50.680704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 11:01:50.947578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684628-90e8-466a-a0d8-959bce3d4c3c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:50.960087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684628-90e8-466a-a0d8-959bce3d4c3c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:50.964310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bc71b-54e6-4760-88ef-538e848aa0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:50.964899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc71b-54e6-4760-88ef-538e848aa0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:50.965178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc71b-54e6-4760-88ef-538e848aa0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:50.965450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc71b-54e6-4760-88ef-538e848aa0cb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:50.973694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bc71b-54e6-4760-88ef-538e848aa0cb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:50.975447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bc71b-54e6-4760-88ef-538e848aa0cb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:50.983921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Aug 18 11:01:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:01:50.989291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:50.998593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b89e1-451e-4115-9903-43619a8a4aa9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:51.004650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba2dbc-de5f-4a55-b272-9c43d6c9c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:51.005289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba2dbc-de5f-4a55-b272-9c43d6c9c484 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:51.005567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba2dbc-de5f-4a55-b272-9c43d6c9c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:51.005839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba2dbc-de5f-4a55-b272-9c43d6c9c484 admin admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:51.007180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba2dbc-de5f-4a55-b272-9c43d6c9c484 admin admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:51.009061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba2dbc-de5f-4a55-b272-9c43d6c9c484 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:51.010603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Aug 18 11:01:50 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:01:52.993288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcea5f8-04e7-4806-b15b-c0ae83efa93c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:52.993834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcea5f8-04e7-4806-b15b-c0ae83efa93c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:52.993834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcea5f8-04e7-4806-b15b-c0ae83efa93c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:52.995186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:01:53.000726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.001221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.001504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.019723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:53.021025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:53.022092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b260d16-a60a-4320-ba6c-53e4c5ace02d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.022380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b260d16-a60a-4320-ba6c-53e4c5ace02d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.022656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b260d16-a60a-4320-ba6c-53e4c5ace02d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.024455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:01:53.029142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.029422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.029692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.071877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG passlib.handlers.bcrypt [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81624) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 11:01:53.072323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG passlib.handlers.bcrypt [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81624) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 11:01:53.103762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:01:53.119415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:53.121019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:53.220837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:01:53.372820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.398235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:53.481928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa00f6-65cc-4663-8160-c48b88a30202 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:53.495042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.516350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:53.516979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.517259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.517534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.528677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:01:53.537893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:01:53.538799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:01:53.539656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:01:53.544714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.598666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692643d-e9f3-4cc9-98ae-97416731bfbc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:53.600981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:53.627729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:53.636832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:53.637681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.637957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.638222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.646687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f348c57-b086-406e-a4d4-4b52d0bf4101 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:53.680656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:53.681254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.681532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.681803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.682471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:01:53.682781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:53.684178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:53.695755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:53.701205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.705948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:53.707499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:53.801981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42d37f-b41e-4a0c-b9c1-8294a2f71d6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:53.804487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:01:53.810025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.813016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:53.836831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:53.837465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.837746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.838020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.839155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:53.840738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:53.842306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:53.848292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.863298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:53.911901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac50a12-fef9-4835-b204-ec262f30d791 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:53.944767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:53.945614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.945893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.946165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.949770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc17da-64c7-4144-bab5-5902171f4d1e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:53.964750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:53.966220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:53.983941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:53.984629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:01:53.984734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:01:53.985601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:01:53.986406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:01:53.986406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:01:53.986406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:01:53.986558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:53.986558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:53.986677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:01:53.986740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:53.987864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:53.989430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:53.991719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:53.995103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:54.051985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:54.079069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:54.097390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e932a-6b03-4a95-8d77-fc05ab903f23 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:54.098676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d0d57-7c4e-4459-8f6e-b13d52d5a5c3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:54.131558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:54.131805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:54.132165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:54.132424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:54.132528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:54.132705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:54.132795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:54.132978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:54.133363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:01:54.133664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:54.133748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.domain.id=default, user.id=f292716c40bd49538840171d62e79f12) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:01:54.133986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:54.135047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb2f0a-47ea-4102-9862-3b03ca33bb6c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:54.135401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529a8de-831b-4e12-a3b4-cacbe8a899a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:54.142548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/role_assignments?user.id=f292716c40bd49538840171d62e79f12&scope.domain.id=default&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:54.155308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:56.450214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4359937-71b2-4221-838b-447da51a4a98 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.451240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4359937-71b2-4221-838b-447da51a4a98 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.451744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4359937-71b2-4221-838b-447da51a4a98 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.453386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:01:56.459177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21754b2-9096-420c-860f-46d0ae856f0b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.459617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21754b2-9096-420c-860f-46d0ae856f0b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.459719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21754b2-9096-420c-860f-46d0ae856f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.461831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a63a-b514-493c-a1b4-32a4d0d7b6ef None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.462125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a63a-b514-493c-a1b4-32a4d0d7b6ef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.462402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a63a-b514-493c-a1b4-32a4d0d7b6ef None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.463857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 13/23] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:01:56.469552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c5ae4-714e-4456-a92e-a9ef82d5237a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.469839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c5ae4-714e-4456-a92e-a9ef82d5237a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.470149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c5ae4-714e-4456-a92e-a9ef82d5237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.481173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a21754b2-9096-420c-860f-46d0ae856f0b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:56.482667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a21754b2-9096-420c-860f-46d0ae856f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:56.501382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d17c5ae4-714e-4456-a92e-a9ef82d5237a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:56.502615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d17c5ae4-714e-4456-a92e-a9ef82d5237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:56.586329 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 11/24] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:01:56.589268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bf446-675d-41bc-864b-7e05b4860a70 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.590487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bf446-675d-41bc-864b-7e05b4860a70 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.591646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bf446-675d-41bc-864b-7e05b4860a70 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.596987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 12/25] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:01:56.604018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b4fa-a85e-485b-a73d-bd9d0f147ed3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.605927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b4fa-a85e-485b-a73d-bd9d0f147ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.607132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b4fa-a85e-485b-a73d-bd9d0f147ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.618133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 14/26] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:01:56.637239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0493b4fa-a85e-485b-a73d-bd9d0f147ed3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:56.638737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0493b4fa-a85e-485b-a73d-bd9d0f147ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:56.700684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:56.735377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 13/27] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:01:56.832588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbe8d0-02c4-4d60-a4f8-c63b0821ef85 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.832897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbe8d0-02c4-4d60-a4f8-c63b0821ef85 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.833147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbe8d0-02c4-4d60-a4f8-c63b0821ef85 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.834776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 15/28] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:01:56.840786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:56.841079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:56.841348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:56.868436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:56.869637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:56.963631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbe8d0-02c4-4d60-a4f8-c63b0821ef85 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:56.963792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 14/29] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:01:57.031366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.132596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:57.182455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbe8d0-02c4-4d60-a4f8-c63b0821ef85 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.222245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd584a9-228c-4c8f-a9ba-184ec1b9ceb3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.230626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:57.249009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:57.249637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:57.249914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:57.250986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:57.263274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:57.264692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:57.264814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:57.265689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:57.266945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:57.267603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:57.268910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:57.270656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:01:57.295662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:01:57.296669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:01:57.298266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:01:57.298266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:01:57.298266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:01:57.304757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 16/30] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:01:57.305628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.421399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 15/31] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Aug 18 11:01:57 2022] POST /identity/v3/roles => generated 219 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:01:57.423416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.424222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc71eb-4d89-4b94-8156-2341e6fcffa2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.454264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:57.464413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:57.465559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:57.466214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:57.466581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:57.479381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:01:57.485460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:01:57.486423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:01:57.487268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:01:57.491659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 17/32] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:01:57.491659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.554976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c060ed-3c49-46f8-be8a-c958afc9fa09 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.607727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:57.608699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:57.609159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:57.610000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:57.610488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33eb92c-5c24-4758-9ec5-888c6a0a1ef4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.611267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:01:57.611753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:57.613799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:57.630438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 16/33] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:01:57.632001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.652101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:57.652825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:57.653186 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:57.653531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:57.654425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:57.656177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:57.690360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:57.752674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6cac0-1a32-4a9c-b3f1-d14a06f3ca45 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.789243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:57.789243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:57.789520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:57.789690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:57.790539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:01:57.790907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:57.792382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:57.801528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 18/34] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Aug 18 11:01:57 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:01:57.802857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.808509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 17/35] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:01:57.819985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:57.886505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:57.956586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8704811-c4ee-4f66-9956-ac0e969951c0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:57.988824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:58.000735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e83f2-a82e-477c-9adf-c701fd2ef1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:58.001486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e83f2-a82e-477c-9adf-c701fd2ef1ff None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.003198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e83f2-a82e-477c-9adf-c701fd2ef1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.003198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e83f2-a82e-477c-9adf-c701fd2ef1ff None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.003198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e83f2-a82e-477c-9adf-c701fd2ef1ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:58.004851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e83f2-a82e-477c-9adf-c701fd2ef1ff None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:58.013005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22b205-6364-4f67-8705-f52a7b2622a6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:58.056761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 19/36] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:01:57 2022] POST /identity/v3/projects => generated 301 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:01:58.058832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41954-ddb7-4526-89b9-33039e954c3e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.059156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41954-ddb7-4526-89b9-33039e954c3e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.059434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41954-ddb7-4526-89b9-33039e954c3e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.060746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 20/37] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:01:58.063195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:58.064154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.064747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.065195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.066973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.067259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.067532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.067799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:58.069623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:58.071591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 18/38] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:01:57 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:01:58.080988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:58.107749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:58.112631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:58.216581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 21/39] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:01:58.244003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fcb8f-b0c3-49f5-af55-8243b3f53215 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:58.280573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56386-ec2d-4775-ad7b-9290a53d9d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:58.281130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56386-ec2d-4775-ad7b-9290a53d9d21 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.281436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56386-ec2d-4775-ad7b-9290a53d9d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.281772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56386-ec2d-4775-ad7b-9290a53d9d21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.283034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56386-ec2d-4775-ad7b-9290a53d9d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:58.284515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56386-ec2d-4775-ad7b-9290a53d9d21 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:58.286119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 19/40] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:58.292491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:58.329202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:58.382716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168eff-73cb-46dd-b089-3bcf4e827c1f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.383032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168eff-73cb-46dd-b089-3bcf4e827c1f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.383283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168eff-73cb-46dd-b089-3bcf4e827c1f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.384915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 20/41] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:01:58.389683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.390227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.390510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.410401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:01:58.410590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b8ce-3575-47bb-b688-0aa1ebf3cf25 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:58.411914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:01:58.430934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:58.448635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:58.449297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.449582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.449856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] PATH_INFO: `/v3/domains/default/users/f292716c40bd49538840171d62e79f12/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.451731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f292716c40bd49538840171d62e79f12, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:58.453310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:58.476309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 22/42] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Aug 18 11:01:58 2022] PUT /identity/v3/domains/default/users/f292716c40bd49538840171d62e79f12/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:01:58.489396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 21/43] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:01:58.499052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:58.705032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:58.720222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595020b-9274-42fa-997d-b7819a38ded0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:58.755033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:58.755644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.755926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.756219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.756887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:58.758277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:58.763553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:58.774959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 23/44] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Aug 18 11:01:58 2022] POST /identity/v3/roles => generated 223 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:01:58.881779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5c3f-05f2-4817-8b75-483648445f5c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:58.923569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76773519-cfda-4938-b73b-65d9da4777ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:58.923569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773519-cfda-4938-b73b-65d9da4777ad None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:58.923569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773519-cfda-4938-b73b-65d9da4777ad None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:58.923935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773519-cfda-4938-b73b-65d9da4777ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:58.925012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76773519-cfda-4938-b73b-65d9da4777ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:58.926518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76773519-cfda-4938-b73b-65d9da4777ad None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:58.928474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 22/45] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:01:58.933881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:01:59.045961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8db73-baac-4eda-bd23-d344cfd8cf38 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:01:59.078992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9567c0-fca5-4d53-8f6a-fa10a54c1f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:01:59.079592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9567c0-fca5-4d53-8f6a-fa10a54c1f6b None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:01:59.079873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9567c0-fca5-4d53-8f6a-fa10a54c1f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:01:59.080146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9567c0-fca5-4d53-8f6a-fa10a54c1f6b None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:01:59.080859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9567c0-fca5-4d53-8f6a-fa10a54c1f6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:01:59.082301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9567c0-fca5-4d53-8f6a-fa10a54c1f6b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:01:59.103208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:01:59.109889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:01:58 2022] POST /identity/v3/projects => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:00.368696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689c71d-b041-413d-bd7d-6a8937f52383 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:00.369187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689c71d-b041-413d-bd7d-6a8937f52383 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:00.369187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689c71d-b041-413d-bd7d-6a8937f52383 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:00.370512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 23/47] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:00.376616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:00.377031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:00.377183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:00.395596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:00.396829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:00.462980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 25/48] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:00.733282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689c71d-b041-413d-bd7d-6a8937f52383 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:00.758762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:00.835610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689c71d-b041-413d-bd7d-6a8937f52383 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:00.871535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9da1-caf6-4449-b919-0ff8613ce79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:00.872460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9da1-caf6-4449-b919-0ff8613ce79a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:00.872739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9da1-caf6-4449-b919-0ff8613ce79a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:00.873007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9da1-caf6-4449-b919-0ff8613ce79a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:00.884235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9da1-caf6-4449-b919-0ff8613ce79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f85a9da1-caf6-4449-b919-0ff8613ce79a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:00.890428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:00.891422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:00.892280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 24/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:00.898865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:00.998040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd9b48-9487-4ffa-91ea-85b2da15119e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:01.026576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ac0-a07b-4d13-b900-979daca2e66b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.026576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ac0-a07b-4d13-b900-979daca2e66b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.026832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ac0-a07b-4d13-b900-979daca2e66b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.028030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 25/50] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:01.029405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:01.034093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.034378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.034654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.036443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:01.037051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.037332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.037605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:01.038763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:01.040425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:01.049858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:01.052953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:01.054164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:01.055144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:01.121055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 26/52] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:01.153134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d003c1-1907-4cf9-8920-1abd6db46dc4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:01.188387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:01.189005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.189286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.189557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.190688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:01.192146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:01.193733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 27/53] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:01 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:01.199902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:01.218869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:01.307526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d090540-c836-4bfe-aa41-d8696a44572c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:01.341679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:01.342301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.342582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.342857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.344665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:01.346125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:01.347844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 27/54] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:01.353754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:01.396328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:01.413102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:01.455415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f8183-e56b-4a8f-a38a-ab757c78ce98 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:01.477443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:01.489624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:01.490230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.490514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.490795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.491515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.domain.id=default, user.id=f292716c40bd49538840171d62e79f12) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:01.491827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:01.493325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:01.496399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf6608-ddc6-4bc3-8694-24c3a716d909 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:01.499163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 28/55] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Aug 18 11:02:01 2022] GET /identity/v3/role_assignments?user.id=f292716c40bd49538840171d62e79f12&scope.domain.id=default&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:01.535133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a52c8-b2d1-4927-9399-a66cbde54c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:01.535787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a52c8-b2d1-4927-9399-a66cbde54c04 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.536072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a52c8-b2d1-4927-9399-a66cbde54c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.536370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a52c8-b2d1-4927-9399-a66cbde54c04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.537520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a52c8-b2d1-4927-9399-a66cbde54c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:01.539001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a52c8-b2d1-4927-9399-a66cbde54c04 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:01.540633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 28/56] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:01.546417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:01.657385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec5a8c-75c2-467b-b722-8cb1918573aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:01.680415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:01.692402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150bc19-af18-443f-b1b3-afe95d880e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:01.693149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150bc19-af18-443f-b1b3-afe95d880e79 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:01.693330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150bc19-af18-443f-b1b3-afe95d880e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:01.693581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150bc19-af18-443f-b1b3-afe95d880e79 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:01.694239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150bc19-af18-443f-b1b3-afe95d880e79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:01.695704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150bc19-af18-443f-b1b3-afe95d880e79 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:01.722201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 29/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Aug 18 11:02:01 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:03.600277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37dd5d-1a3f-4000-bf77-32d7ac68434d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:03.600824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37dd5d-1a3f-4000-bf77-32d7ac68434d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:03.600824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37dd5d-1a3f-4000-bf77-32d7ac68434d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:03.602107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 29/58] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:03.607169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:03.607584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:03.607724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:03.626709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:03.627914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:03.692948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 30/59] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:03.963819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37dd5d-1a3f-4000-bf77-32d7ac68434d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:04.065126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37dd5d-1a3f-4000-bf77-32d7ac68434d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:04.097936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288bac42-6506-438d-a34f-58d9323db362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:04.098566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bac42-6506-438d-a34f-58d9323db362 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:04.098843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bac42-6506-438d-a34f-58d9323db362 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:04.099110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bac42-6506-438d-a34f-58d9323db362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:04.100249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288bac42-6506-438d-a34f-58d9323db362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:04.101670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288bac42-6506-438d-a34f-58d9323db362 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:04.103210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 30/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:04.108749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:04.206564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470709d-b678-4fdb-a341-894cfb112487 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:04.214010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:04.241136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae98412-95a8-44c2-a4af-76730eaf9180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:04.241739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae98412-95a8-44c2-a4af-76730eaf9180 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:04.242021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae98412-95a8-44c2-a4af-76730eaf9180 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:04.242297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae98412-95a8-44c2-a4af-76730eaf9180 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:04.242932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae98412-95a8-44c2-a4af-76730eaf9180 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:04.244449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae98412-95a8-44c2-a4af-76730eaf9180 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:04.270205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 31/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:04 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:07.216904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63f8cd-e1fb-4f47-a798-c7a6bfd3961b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.217411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63f8cd-e1fb-4f47-a798-c7a6bfd3961b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.217482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63f8cd-e1fb-4f47-a798-c7a6bfd3961b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.219624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 31/62] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:07.238165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0a9-a495-45db-ad85-c277f7c8296c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.238657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0a9-a495-45db-ad85-c277f7c8296c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.238758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0a9-a495-45db-ad85-c277f7c8296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.265248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e347a0a9-a495-45db-ad85-c277f7c8296c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:07.267093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e347a0a9-a495-45db-ad85-c277f7c8296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:07.349811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 32/63] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:07.407856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa67a8-fc82-4966-b4ae-b36fc5174919 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.408172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa67a8-fc82-4966-b4ae-b36fc5174919 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.408445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa67a8-fc82-4966-b4ae-b36fc5174919 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.409816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 32/64] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:07.425835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfb037-7395-42f0-938a-0d728e2a8b8c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.426116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfb037-7395-42f0-938a-0d728e2a8b8c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.426402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfb037-7395-42f0-938a-0d728e2a8b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.457258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dbcfb037-7395-42f0-938a-0d728e2a8b8c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:07.458483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dbcfb037-7395-42f0-938a-0d728e2a8b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:07.547976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 33/65] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:07.548328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f61f6-96d4-4f53-a91f-8d69b6356de0 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.548874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f61f6-96d4-4f53-a91f-8d69b6356de0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.549345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f61f6-96d4-4f53-a91f-8d69b6356de0 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.550906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 33/66] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:07.566672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a80dd-b678-4da5-a4af-3874e84f01be None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.566956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a80dd-b678-4da5-a4af-3874e84f01be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.567239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a80dd-b678-4da5-a4af-3874e84f01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.596534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6a4a80dd-b678-4da5-a4af-3874e84f01be None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:07.597735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6a4a80dd-b678-4da5-a4af-3874e84f01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:07.637042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:07.693684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 34/67] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:07.742477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f61f6-96d4-4f53-a91f-8d69b6356de0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:07.768775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:07.882593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213e89-3efe-4ea5-969a-4d54920dcad9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.883160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213e89-3efe-4ea5-969a-4d54920dcad9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.883638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213e89-3efe-4ea5-969a-4d54920dcad9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.885197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 35/68] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:07.889820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f61f6-96d4-4f53-a91f-8d69b6356de0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:07.894132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.894415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.894687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.931209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:07.932670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:07.947887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:07.948551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:07.948835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:07.949108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:07.965426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:07.971173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:07.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:07.972793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:07.973430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.020822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 36/70] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:08.021977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.069578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.095096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0865a85-4975-4e4e-805c-090ac43a6928 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.111824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.132106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca301-9c20-4769-b99e-3becd4d8159c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.133043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb124a4-034a-4777-8486-87f1f213353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.133674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb124a4-034a-4777-8486-87f1f213353e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.133959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb124a4-034a-4777-8486-87f1f213353e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.134234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb124a4-034a-4777-8486-87f1f213353e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.145674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb124a4-034a-4777-8486-87f1f213353e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-bfb124a4-034a-4777-8486-87f1f213353e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:08.150478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:08.151302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:08.152051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 35/71] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:08.153432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7218e4-1dfb-41ed-a7db-501e7141efa3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.153432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7218e4-1dfb-41ed-a7db-501e7141efa3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.153432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7218e4-1dfb-41ed-a7db-501e7141efa3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.159362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 36/72] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:08.159362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7218e4-1dfb-41ed-a7db-501e7141efa3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.175761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.176412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.176701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.176974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.177645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:08.177949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.179327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819687eb-0653-49e5-984c-e37aeeb3fc81 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:08.187686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 37/73] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:08.189050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.189336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.189622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.207816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:08.209034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:08.271689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7218e4-1dfb-41ed-a7db-501e7141efa3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.279673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 38/74] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:08.288702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.316198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.316844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.317130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.317403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.320444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.330941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:08.335853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:08.336864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:08.337669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:08.340811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 37/75] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:08.340811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.369861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.402888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606452d7-036f-48ed-916a-2a7416388db9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.441576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.442296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.442581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.442974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.443641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:08.443947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.445372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:08.445910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d425c3d-d08c-449e-8e8c-5f109031c178 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.458345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 39/76] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:08.458345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.567740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f34fb5-3a50-4eba-9282-69afd57250f5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.587076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.587702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.587985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.588295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.602109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.602734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.603017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.603297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.603781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.604444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.605303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:08.605888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:08.606930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 38/77] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:08.608234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 40/78] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:08.608362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.608438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.616268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.670792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.715445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b7610-25d9-40e6-8891-bdc5a2244e1d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.720497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbf53-5d01-4c50-91fa-96f5db24c06f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.749624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.750595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.750882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.751157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.751824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:08.752130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.753637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:08.753788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.754416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.754696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.754968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.756092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.757546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:08.759114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 41/79] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:08.764087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.767282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 39/80] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:08.768107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.830334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.861896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6103e24-fd86-466a-9b7e-b4af29813f3f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.869539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21962d5-ab94-4708-83e3-951f994c74c3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:08.885422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:08.895522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.896147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.896459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.896738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.903869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:08.904505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:08.904784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:08.905055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:08.907495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:08.912689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:08.913631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:08.914411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 42/81] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:08.915793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:08.974069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:08.975459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.023535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfa3da-aa3c-4ece-b3cf-66fb6341663d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.052375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.057469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.058079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.058362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.058640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.059285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.060803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.081294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 43/82] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:08 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:09.082114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.090674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 40/83] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:09.091796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.112785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.206359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec2c08-3cdf-4fe4-a120-0637227c8b0d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.208852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38084cd8-f840-4efa-a6d5-fcaed7301903 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.247478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.248018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.248328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.248603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.249211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.250451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.250831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.251210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.253507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.255081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.256838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 41/84] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:09.261460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.310423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.328477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.330234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.367176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.383655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce4946-f8bd-458a-925d-981bb4936e44 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.431436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.432849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.433362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.434207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.435404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:09.435915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.438104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.449264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 42/85] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:09.450735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.459925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 44/86] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:09.460945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.558021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.571549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b383e-37c2-4bd6-bee5-e8a376144d98 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.592692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5de61-8849-4cf8-999d-065d36fe38b4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.613824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.614498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.614791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.615068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.634296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.634990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.635277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.635602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.637457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.638914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.640634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 43/87] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:09.641489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.661690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.703086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.705259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.761734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149baf5-c85b-4b9f-9438-ef3f693255bc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.804224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.805257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.805540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.805817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.807348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.809204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.812346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 44/88] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:09.813371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.826297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.841278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 45/89] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:09.847205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:09.902950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:09.922723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c141ba7-5710-47bb-9872-b23b69ffdae1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.959186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:09.960147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:09.960405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:09.960687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:09.961861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:09.962571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628b392-1b2a-4730-b246-e6472c392599 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:09.963438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:09.966653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 45/90] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:09.967457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:10.002554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:10.003548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:10.004551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:10.005292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:10.006486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=b42994e086454bed9832f0d860d67c67) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:10.007484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:10.009701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:10.017540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 46/91] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/role_assignments?user.id=b42994e086454bed9832f0d860d67c67&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:10.025090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:10.136904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:10.187257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d2e26-94df-43ba-a48f-34886a9fd700 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:10.198136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93fc1b-1dc7-443c-8550-226819fb11eb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:10.232164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:10.233168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:10.233641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:10.234179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:10.241283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:10.241922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:10.242204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:10.242477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:10.243650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:10.245664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:10.247989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 47/92] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:10 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:10.253285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:10.277280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:10.370174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa25cc-e28d-47e2-a06d-04cdf21e9d78 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:10.397124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:10.399306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:10.408217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:10.420342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:10.421330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:10.421676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:10.421950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:10.422690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=f292716c40bd49538840171d62e79f12) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:10.423001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:10.424886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:10.431149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 48/93] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:10 2022] GET /identity/v3/role_assignments?user.id=f292716c40bd49538840171d62e79f12&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:10.431811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:10.557645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f50e6-b047-4ee3-8a65-1084b49e56a8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:10.600224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:10.600224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:10.600224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:10.600224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:10.600612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] RBAC: Adding query filter params (role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=b42994e086454bed9832f0d860d67c67) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:10.600612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:10.601636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:10.608559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 49/94] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:10 2022] GET /identity/v3/role_assignments?user.id=b42994e086454bed9832f0d860d67c67&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:10.628661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 46/95] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:10.634951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:10.698688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:10.757478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dc2c5-fa70-4349-b46c-5c8e8d09ab33 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:10.795054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562758be-b1b1-4253-8934-f1e3f6266a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:10.795701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562758be-b1b1-4253-8934-f1e3f6266a98 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:10.795983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562758be-b1b1-4253-8934-f1e3f6266a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:10.796311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562758be-b1b1-4253-8934-f1e3f6266a98 None admin] PATH_INFO: `/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:10.797670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562758be-b1b1-4253-8934-f1e3f6266a98 None admin] RBAC: Authorizing `identity:get_project(project_id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:10.799137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562758be-b1b1-4253-8934-f1e3f6266a98 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:10.800835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 50/96] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:10 2022] GET /identity/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:10.806848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:10.898777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:10.960578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6ad91-37b8-4be2-941a-2d76562b8d85 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:11.004440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:11.005422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:11.005893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:11.006367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:11.007280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3, user.id=b42994e086454bed9832f0d860d67c67) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:11.007797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:11.009540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed40806-4aec-4c4c-8257-0e33d081ac57 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:11.023001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 47/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:10 2022] GET /identity/v3/role_assignments?user.id=b42994e086454bed9832f0d860d67c67&scope.project.id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:11.371358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bbf59-f28d-44ba-b5cf-3e2f0298d8e8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:11.372095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bbf59-f28d-44ba-b5cf-3e2f0298d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:11.372095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bbf59-f28d-44ba-b5cf-3e2f0298d8e8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:11.373694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 51/98] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:11.379118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910d98a-e3e5-4c08-b253-df39f5d71e96 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:11.379408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910d98a-e3e5-4c08-b253-df39f5d71e96 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:11.379686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910d98a-e3e5-4c08-b253-df39f5d71e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:11.405239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7910d98a-e3e5-4c08-b253-df39f5d71e96 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:11.406999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7910d98a-e3e5-4c08-b253-df39f5d71e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:11.494464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 48/99] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:11.767344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccf90b-4ee5-4fcf-99f2-1eba8c3d232d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:11.767710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccf90b-4ee5-4fcf-99f2-1eba8c3d232d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:11.767903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccf90b-4ee5-4fcf-99f2-1eba8c3d232d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:11.769297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 52/100] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:11.774645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:11.774947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:11.775228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:11.797735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:11.799230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:11.844614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:11.875245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 49/101] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:12.012800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ccf90b-4ee5-4fcf-99f2-1eba8c3d232d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:12.137836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ccf90b-4ee5-4fcf-99f2-1eba8c3d232d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:12.160544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:12.183283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:12.183917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:12.184212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:12.184499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:12.188861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:12.190333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:12.192227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 53/102] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:12.197823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:12.330092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a056706-d624-40d2-82fd-2970507c1419 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:12.334478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:12.392712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0542706-9522-46b3-a14f-446681268423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:12.393808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542706-9522-46b3-a14f-446681268423 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:12.394284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542706-9522-46b3-a14f-446681268423 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:12.394750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542706-9522-46b3-a14f-446681268423 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:12.395573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0542706-9522-46b3-a14f-446681268423 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:12.397344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0542706-9522-46b3-a14f-446681268423 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:12.458784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 50/103] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:02:12 2022] POST /identity/v3/projects => generated 302 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:12.462282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfc985-e9fa-418f-b5bd-ccac381e6f35 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:12.524151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:12.524828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:12.525114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:12.525388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:12.542637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:12.549331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:12.550289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:12.551028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 54/104] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:12.556476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0542706-9522-46b3-a14f-446681268423 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:12.606545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:12.720358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0542706-9522-46b3-a14f-446681268423 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:12.757338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:12.757979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:12.758261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:12.758536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:12.759226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:12.759534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:12.760978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:12.771014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 51/105] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:12.776163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:12.848715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:12.918066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228a40a-d598-4271-b0da-b16fd32b8398 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:12.965530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b12534-4858-488f-b109-1ba3468466a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:12.966172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12534-4858-488f-b109-1ba3468466a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:12.966457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12534-4858-488f-b109-1ba3468466a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:12.966735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12534-4858-488f-b109-1ba3468466a0 None admin] PATH_INFO: `/v3/users/15ff1b46ca5c46c0bc07b4034bf5f512` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:12.984902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b12534-4858-488f-b109-1ba3468466a0 None admin] RBAC: Authorizing `identity:get_user(user_id=15ff1b46ca5c46c0bc07b4034bf5f512)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:12.986385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b12534-4858-488f-b109-1ba3468466a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:12.988253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 55/106] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:12 2022] GET /identity/v3/users/15ff1b46ca5c46c0bc07b4034bf5f512 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:12.993734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:13.119029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:13.133426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba677fdd-1480-4bf4-b485-794ab86e364b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:13.174942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:13.175478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.175758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.176030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.209737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b42f-9aae-43a7-8d91-8fb3e1dfbdbe None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.210369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b42f-9aae-43a7-8d91-8fb3e1dfbdbe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.210873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b42f-9aae-43a7-8d91-8fb3e1dfbdbe None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.212446 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 56/107] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:13.221572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.222146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.222622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.250130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:13.251815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:13.261266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:13.262776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:13.355976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:13.369798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 52/108] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:13.375973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:13.378597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 57/109] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:13.519145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ac41c-461a-4741-9123-a22fbd974db3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:13.555249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70850f2-b6a1-47d6-93f6-e862b35c4aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:13.556209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70850f2-b6a1-47d6-93f6-e862b35c4aee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.556711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70850f2-b6a1-47d6-93f6-e862b35c4aee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.557151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70850f2-b6a1-47d6-93f6-e862b35c4aee None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.559906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70850f2-b6a1-47d6-93f6-e862b35c4aee None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:13.561575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70850f2-b6a1-47d6-93f6-e862b35c4aee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:13.564596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 53/110] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:13 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:13.570459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:13.618579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:13.648915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62797f-0b31-4828-b1c9-cb68d91b3fb8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.649217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62797f-0b31-4828-b1c9-cb68d91b3fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.649463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62797f-0b31-4828-b1c9-cb68d91b3fb8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.650944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 54/111] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:13.658287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.659255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.660014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.685600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:13.686808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:13.708436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac405e2-9134-43e0-a8ec-b99a26f0834f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:13.746101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:13.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.747816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.748376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.749501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] RBAC: Adding query filter params (role.id=334460cecea042219ed608f1b5ff9096, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=15ff1b46ca5c46c0bc07b4034bf5f512) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:13.750583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:13.752233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c426216-2a7b-41b3-a664-ecae1db03d10 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:13.758413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 58/112] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:13 2022] GET /identity/v3/role_assignments?user.id=15ff1b46ca5c46c0bc07b4034bf5f512&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=334460cecea042219ed608f1b5ff9096 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:13.759979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89c4a4-d794-4173-9153-147dc6c9be16 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.760493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89c4a4-d794-4173-9153-147dc6c9be16 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.760978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89c4a4-d794-4173-9153-147dc6c9be16 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.762590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 59/113] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:13.769578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceba7e-e4ab-40f2-9e21-ae25872e54d8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.770171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceba7e-e4ab-40f2-9e21-ae25872e54d8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.770640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceba7e-e4ab-40f2-9e21-ae25872e54d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.805878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:13.810024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-98ceba7e-e4ab-40f2-9e21-ae25872e54d8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:13.811826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-98ceba7e-e4ab-40f2-9e21-ae25872e54d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:13.849779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 55/114] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:13.850874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:13.952420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 60/115] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:13.954373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d0f81-1aad-470b-bd6f-2a11fc8a0607 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.954659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d0f81-1aad-470b-bd6f-2a11fc8a0607 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.954934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d0f81-1aad-470b-bd6f-2a11fc8a0607 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:13.956548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 61/116] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:13.962676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:13.962959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:13.963241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.009205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:14.010410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:14.014203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deb20c-18b6-4e17-aa87-f74fc81f6499 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.051219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:14.082853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.084003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.084523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.085090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.105374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:14.109464 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 62/117] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:14.111581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:14.112445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:14.113238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:14.120144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 56/118] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:14.121332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.231900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4891e1-6335-4d6f-a7c7-8fdbfe358d70 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.269182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.269418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.269647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.269911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.270575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:14.270875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:14.272276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:14.280856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 63/119] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:14.286723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.314418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.349322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:14.382354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:14.480118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329d998-ee62-4ef8-9d04-1750f3cff61c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.533807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.534855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.535154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.535416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.537626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:14.539585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:14.542046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 57/120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:14.542842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.551064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53909-bfb4-4f1c-a751-f0da6be92135 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.601275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.602734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.603314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.603787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.620006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:14.628172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:14.629049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:14.629801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:14.632834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 64/121] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:14.634170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.672557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635faab1-08f5-46cf-bb29-583f799db5e7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.692088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:14.716973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.718094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.718094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.718394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.730728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:14.737444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:14.738659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 58/122] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:14.739418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.761897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:14.799034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96335252-5cab-448a-b8b8-86271130b61d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.844069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.845515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.846026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.846514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.858594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:14.866404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e3007-638a-4686-9802-30a8ff62a7d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:14.867311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:14.868981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:14.870564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:14.880781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 65/123] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:14.882646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.909226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:14.909226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:14.909775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:14.910190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:14.917477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:14.917477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:14.917477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 59/124] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:14.923971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:14.981096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.048924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710a79f-c596-47cb-a86a-2a7489c79572 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.053275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e8330-3874-440c-80da-2f3d42d55792 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.087772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.088668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.088860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.089104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.089779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:15.090081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.091550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.104549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 60/125] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.105335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.114485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.115118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.115827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.115827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.117138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:15.117456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.118901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.127764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.135749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 66/126] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.137499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.222636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ffbf4-e1f7-4ef7-99d4-1a7e18e289e0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.225265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.260700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.261485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.261936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.262276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.263218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:15.263477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.265272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.266987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81215a-83bb-4f55-aab0-30f82003cb0d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.274613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 61/127] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.275892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.307720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.308781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.309252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.309790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/b42994e086454bed9832f0d860d67c67/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.312106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d671114d0af4f33841a2dc83ed32dc3, user_id=b42994e086454bed9832f0d860d67c67, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.314089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.330063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 67/128] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:15 2022] PUT /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/b42994e086454bed9832f0d860d67c67/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:15.331194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.400720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729cce-b1f0-4528-bc1f-d63c985fa301 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.401846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.445193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.446233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.446785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.447242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.448780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.450692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.453276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 62/129] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.453878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba440c74-97dc-4bc2-8f4f-751a2c5e3c22 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.454800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.495915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.498470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.498848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.499131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.499532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.502238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.503753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.505493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 68/130] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.506881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.592570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d5504-4f41-44cf-abd8-c452400a93fb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.623885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7c9d-33cf-4cb1-a8ef-115119eb08fa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.648961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.649618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.649908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.650192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.651394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.652879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.654781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 63/131] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.655568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.656356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.661727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.662535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.663009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.663416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.664752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.666829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.669019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 69/132] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.669937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.777960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20c066-698f-4acb-a764-f70181605faf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.784336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf0bbf-88ac-417e-9015-56d93609ace6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.785917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.817910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.818873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.819291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.819697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] PATH_INFO: `/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.821200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] RBAC: Authorizing `identity:get_project(project_id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.821853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd4727-45b2-421c-82bb-060e306fcd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.822563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd4727-45b2-421c-82bb-060e306fcd46 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.822923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd4727-45b2-421c-82bb-060e306fcd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.823262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.823381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd4727-45b2-421c-82bb-060e306fcd46 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.824577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd4727-45b2-421c-82bb-060e306fcd46 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.825366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 70/133] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.826171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd4727-45b2-421c-82bb-060e306fcd46 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.826387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.827985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 64/134] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:15 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:15.829267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942795-8d47-425f-8c18-338543ab1f51 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.829759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942795-8d47-425f-8c18-338543ab1f51 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.830135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942795-8d47-425f-8c18-338543ab1f51 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.831641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 65/135] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:15.836322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59942795-8d47-425f-8c18-338543ab1f51 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:15.896126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:15.940522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef88b4b-303d-442b-9519-41dea5cda17d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.950423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59942795-8d47-425f-8c18-338543ab1f51 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:15.978365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.979356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.979801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.980230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/b42994e086454bed9832f0d860d67c67/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.982776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] RBAC: Authorizing `identity:create_grant(project_id=8d671114d0af4f33841a2dc83ed32dc3, user_id=b42994e086454bed9832f0d860d67c67, role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.985116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:15.988006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81520c9b-ec42-4e4e-9536-3e876efcecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:15.989003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81520c9b-ec42-4e4e-9536-3e876efcecfe None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:15.989387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81520c9b-ec42-4e4e-9536-3e876efcecfe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:15.989767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81520c9b-ec42-4e4e-9536-3e876efcecfe None admin] PATH_INFO: `/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3/users/b42994e086454bed9832f0d860d67c67/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:15.992013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81520c9b-ec42-4e4e-9536-3e876efcecfe None admin] RBAC: Authorizing `identity:create_grant(project_id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3, user_id=b42994e086454bed9832f0d860d67c67, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:15.993539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81520c9b-ec42-4e4e-9536-3e876efcecfe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:16.001702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 71/136] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:15 2022] PUT /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/b42994e086454bed9832f0d860d67c67/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:16.002862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:16.035964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 66/137] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:15 2022] PUT /identity/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3/users/b42994e086454bed9832f0d860d67c67/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:16.037381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a8925-8372-446b-a633-c8429dac4c7f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:16.037732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a8925-8372-446b-a633-c8429dac4c7f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:16.038017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a8925-8372-446b-a633-c8429dac4c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:16.089536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-003a8925-8372-446b-a633-c8429dac4c7f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:16.091270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-003a8925-8372-446b-a633-c8429dac4c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:16.127306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2b2c-a1a5-4585-b2bc-608417a6817a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:16.134649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:16.161437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:16.174968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:16.176094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:16.176704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:16.177296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/f292716c40bd49538840171d62e79f12/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:16.179770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] RBAC: Authorizing `identity:create_grant(project_id=8d671114d0af4f33841a2dc83ed32dc3, user_id=f292716c40bd49538840171d62e79f12, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:16.181682 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:16.194323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 67/138] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:16.202019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5325507-aca2-4be2-90f2-4937770a199e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:16.202375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5325507-aca2-4be2-90f2-4937770a199e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:16.203218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5325507-aca2-4be2-90f2-4937770a199e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:16.205252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 68/139] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:16.220297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:16.220587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:16.220806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 72/140] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:16 2022] PUT /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/f292716c40bd49538840171d62e79f12/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:16.220910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:16.261367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:16.262605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:16.350356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 69/141] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:16.542497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:16.664528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:16.682904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf022e5-0255-4906-a769-f80875c80c1c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:16.762904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:16.763551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:16.763822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:16.764100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:16.769179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:16.770403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:16.772021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 73/142] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:16.793193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:16.868504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:16.899409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d2436-7ddc-4a90-beb1-84d1e6fe5142 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:16.945082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:17.019779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:17.019779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.020182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.020676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.051939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41f4d2-4f16-4518-8d8a-9d33fb3ad336 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:17.061182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:17.066546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:17.067383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:17.068151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 70/143] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:17.073597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:17.123972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:17.125610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.126038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.126538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.127325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:17.135508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:17.183835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 74/144] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:16 2022] POST /identity/v3/users => generated 283 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:02:17.268760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f82268-01c6-4460-86f6-9a953ec38130 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:17.321631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:17.322181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.322463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.322725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.323417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:17.323724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:17.325179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:17.334356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 71/145] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:17.340285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:17.490261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023228c9-4050-4b2c-bba0-3fc59066c177 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:17.501175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:17.531614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5274-6915-410f-b18f-51b4832559af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:17.532301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5274-6915-410f-b18f-51b4832559af None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.532583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5274-6915-410f-b18f-51b4832559af None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.532857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5274-6915-410f-b18f-51b4832559af None admin] PATH_INFO: `/v3/users/15ff1b46ca5c46c0bc07b4034bf5f512` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.534138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df5274-6915-410f-b18f-51b4832559af None admin] RBAC: Authorizing `identity:get_user(user_id=15ff1b46ca5c46c0bc07b4034bf5f512)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:17.535596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df5274-6915-410f-b18f-51b4832559af None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:17.537210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 75/146] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:17 2022] GET /identity/v3/users/15ff1b46ca5c46c0bc07b4034bf5f512 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:17.542951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:17.564609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:17.675213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d25681-4a44-4f89-9e50-a24cf7e64723 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:17.715063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ab769-491f-4fd5-9402-6fcaf9fa7f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:17.715703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab769-491f-4fd5-9402-6fcaf9fa7f9b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.715987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab769-491f-4fd5-9402-6fcaf9fa7f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.716287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab769-491f-4fd5-9402-6fcaf9fa7f9b None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.717450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ab769-491f-4fd5-9402-6fcaf9fa7f9b None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:17.719736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ab769-491f-4fd5-9402-6fcaf9fa7f9b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:17.721426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 72/147] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:17 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:17.727301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5274-6915-410f-b18f-51b4832559af None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:17.796912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfb4b-78ad-445f-a6b9-a0cd9b9d47c5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.797257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfb4b-78ad-445f-a6b9-a0cd9b9d47c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.797476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfb4b-78ad-445f-a6b9-a0cd9b9d47c5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.798963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 73/148] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:17.804093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.804406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.804687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.835344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:17.836626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:17.875494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5274-6915-410f-b18f-51b4832559af None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:17.920181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:17.920923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:17.921338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:17.921614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/15ff1b46ca5c46c0bc07b4034bf5f512/roles/334460cecea042219ed608f1b5ff9096` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:17.923574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=8d671114d0af4f33841a2dc83ed32dc3, user_id=15ff1b46ca5c46c0bc07b4034bf5f512, role_id=334460cecea042219ed608f1b5ff9096)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:17.925049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:17.950175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 74/149] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:17.950439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 76/150] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:17 2022] PUT /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/users/15ff1b46ca5c46c0bc07b4034bf5f512/roles/334460cecea042219ed608f1b5ff9096 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:18.340846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:18.353451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:18.467002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617fd2-f130-4b27-b6ee-33483aeb7e8c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:18.511398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:18.512030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:18.512332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:18.512606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:18.529500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:18.535245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:18.536088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:18.536895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 75/151] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:18.545427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:18.596588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:18.679761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43ed08-ee24-421f-a056-f8a7a5306eaf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:18.735793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:18.736766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:18.736766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:18.737076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:18.737680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:18.737980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:18.739356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:18.751739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 77/152] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:18.757283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:18.860899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:18.901250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761135-5a74-435d-a20a-679185f8eaa6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:18.943692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b1ca3-7e83-4515-945e-03d094370d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:18.944366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b1ca3-7e83-4515-945e-03d094370d45 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:18.944649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b1ca3-7e83-4515-945e-03d094370d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:18.944920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b1ca3-7e83-4515-945e-03d094370d45 None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:18.946110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b1ca3-7e83-4515-945e-03d094370d45 None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:18.947582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b1ca3-7e83-4515-945e-03d094370d45 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:18.949387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 76/153] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:18 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:18.955187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:19.123494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0267313-9987-4b6f-a940-d6afac9adc4f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.123781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0267313-9987-4b6f-a940-d6afac9adc4f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.123957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0267313-9987-4b6f-a940-d6afac9adc4f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.125768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 77/154] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:19.133416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:19.133595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcae35-d4f3-4228-aa51-2fce1b63b6d8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.133791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcae35-d4f3-4228-aa51-2fce1b63b6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.134063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcae35-d4f3-4228-aa51-2fce1b63b6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.137397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8dcdf-c23f-477e-a1c7-a399da89acc9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:19.166767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0ebcae35-d4f3-4228-aa51-2fce1b63b6d8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:19.167968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0ebcae35-d4f3-4228-aa51-2fce1b63b6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:19.205851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bac1c-b676-4bfa-8bcf-8e85ce677342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:19.206470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bac1c-b676-4bfa-8bcf-8e85ce677342 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.206749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bac1c-b676-4bfa-8bcf-8e85ce677342 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.207015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bac1c-b676-4bfa-8bcf-8e85ce677342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.302461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 78/155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:19.303564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d96b4-f3bd-4345-9b42-f520492442bd None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.303850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d96b4-f3bd-4345-9b42-f520492442bd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.304129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d96b4-f3bd-4345-9b42-f520492442bd None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.305605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 79/156] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:19.320584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.320883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.321129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.330544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bac1c-b676-4bfa-8bcf-8e85ce677342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:19.332030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bac1c-b676-4bfa-8bcf-8e85ce677342 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:19.350797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:19.351991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:19.482756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:19.490138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 80/157] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:19.512298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:19.516283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 78/158] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:19.522229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:19.629502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19c76-03ce-40a1-8bbb-7bdba2faa430 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:19.635208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72018b-93df-40d6-98bd-aa476e2d5771 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.635494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72018b-93df-40d6-98bd-aa476e2d5771 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.635770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72018b-93df-40d6-98bd-aa476e2d5771 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.637320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 79/159] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:19.642711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.642985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.643261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.677182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:19.681924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:19.715029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:19.715677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:19.715951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:19.716236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:19.718118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:19.719636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:19.723048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 81/160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:19 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:19.729543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:19.838740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 80/161] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 18 11:02:19.853759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:19.904007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:19.923531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe32134-48fb-4a0b-bb1c-5b52af306af1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.000832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.001447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.001724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.001987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=b42994e086454bed9832f0d860d67c67) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:20.003018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:20.004486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:20.018781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 82/162] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:19 2022] GET /identity/v3/role_assignments?user.id=b42994e086454bed9832f0d860d67c67&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:20.019935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:20.205915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e964bb0-5fd0-4369-9bf6-8342634f25ef None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.257970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:20.266498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.267160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.267444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.267724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.267884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dfe6b-b1ea-4dc3-a19f-ca6556373d7a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.295343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:20.301049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:20.301963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:20.302747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 81/163] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:20.303479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:20.352568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:20.379249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.379907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.380203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.380484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.408786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:20.428563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:20.429495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:20.430300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 83/164] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:20.430946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:20.456111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db442-c4ad-4716-87de-9ebdf5c1a623 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.506645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.507434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.507713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.508014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.539909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:20.544734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:20.545810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:20.547252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 82/165] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:20.548011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:20.618374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d0adc-b50f-4cdd-8e58-bfef077ce29e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.648004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:20.687713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.688360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.688644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.688919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.689588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:20.689894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:20.691283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:20.712161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 84/166] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:20.713001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:20.786988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233921c-b496-421c-9426-765ca689d4f0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.804607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:20.861604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8b561-23c7-4c64-b869-0a69374d3673 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:20.889532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.890741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.890995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.891293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.892041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:20.892371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:20.893870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:20.903167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 83/167] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:20.903767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:20.904016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:20.904510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:20.904801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:20.905074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:20.906389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:20.906698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:20.908083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:20.917513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 85/168] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:20.918443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:21.099653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9f520-130e-4a88-9553-8f0c290eb688 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:21.119695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:21.126933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87cff6-213b-4429-8cf4-865c1676e46c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:21.143583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:21.165617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:21.166333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:21.166678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:21.167010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:21.168231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:21.170019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:21.171864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 86/169] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:21.172423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:21.200893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:21.201540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:21.201819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:21.202089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] PATH_INFO: `/v3/users/f292716c40bd49538840171d62e79f12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:21.204062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] RBAC: Authorizing `identity:get_user(user_id=f292716c40bd49538840171d62e79f12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:21.205546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:21.207288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 84/170] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:20 2022] GET /identity/v3/users/f292716c40bd49538840171d62e79f12 => generated 247 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:21.207890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:21.366596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dc6f3-8182-480f-bdbd-9185bc98dd7b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:21.409831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:21.410443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:21.410718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:21.410986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:21.476584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:21.487866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bef00-57f1-4315-9623-5f768970164a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:21.489848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:21.491341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:21.551074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:21.551702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:21.551975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:21.552275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] PATH_INFO: `/v3/users/b42994e086454bed9832f0d860d67c67` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:21.554316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] RBAC: Authorizing `identity:get_user(user_id=b42994e086454bed9832f0d860d67c67)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:21.555762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:21.560471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 87/171] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:21 2022] GET /identity/v3/users/b42994e086454bed9832f0d860d67c67 => generated 275 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:21.561260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:21.651485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 85/172] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:21.652084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:21.656824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:21.742704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:21.750422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50102a-46ae-4489-a2fd-7d3b5f8c08ab None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:21.768005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dd281-4dcf-4b89-a830-a6720ed12919 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:21.806597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597c19-2a75-44d5-9254-04e4a5b4d115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:21.807524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597c19-2a75-44d5-9254-04e4a5b4d115 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:21.808169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597c19-2a75-44d5-9254-04e4a5b4d115 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:21.809956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597c19-2a75-44d5-9254-04e4a5b4d115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:21.830564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6670-e08f-4685-9637-55f176cc24f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:21.831460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6670-e08f-4685-9637-55f176cc24f2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:21.831958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6670-e08f-4685-9637-55f176cc24f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:21.832416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6670-e08f-4685-9637-55f176cc24f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:21.942873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597c19-2a75-44d5-9254-04e4a5b4d115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:21.944570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597c19-2a75-44d5-9254-04e4a5b4d115 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:21.959694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6670-e08f-4685-9637-55f176cc24f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:21.962303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6670-e08f-4685-9637-55f176cc24f2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.031059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.033660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.059753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 86/173] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:22.064715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de58c9-f853-4749-875e-1d0b0dce4830 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.064715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de58c9-f853-4749-875e-1d0b0dce4830 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.064715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de58c9-f853-4749-875e-1d0b0dce4830 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.064715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 87/174] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:22.065427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de58c9-f853-4749-875e-1d0b0dce4830 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.084221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 88/175] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:22.089994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575690-581a-4f21-a7f0-27ce3bfdff32 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.090490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575690-581a-4f21-a7f0-27ce3bfdff32 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.090942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575690-581a-4f21-a7f0-27ce3bfdff32 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.096235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 89/176] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:22.097678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad67622-b973-4db1-9441-85816c170245 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.097977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad67622-b973-4db1-9441-85816c170245 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.098219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad67622-b973-4db1-9441-85816c170245 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.099754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 90/177] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:22.100906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad67622-b973-4db1-9441-85816c170245 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.177959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de58c9-f853-4749-875e-1d0b0dce4830 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:22.205108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad67622-b973-4db1-9441-85816c170245 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:22.212362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d04d53-5f33-4e92-aa43-1fabbf337e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:22.212989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04d53-5f33-4e92-aa43-1fabbf337e93 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.213269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04d53-5f33-4e92-aa43-1fabbf337e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.213540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04d53-5f33-4e92-aa43-1fabbf337e93 None admin] PATH_INFO: `/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.214679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d04d53-5f33-4e92-aa43-1fabbf337e93 None admin] RBAC: Authorizing `identity:get_project(project_id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:22.216131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d04d53-5f33-4e92-aa43-1fabbf337e93 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.217782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 88/178] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/projects/0e51f0f7a2604d7fb35d5ab7dc4dc8e3 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:22.219308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c71759-4bc8-4349-9cb1-3cc4730ddf60 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.219605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c71759-4bc8-4349-9cb1-3cc4730ddf60 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.219864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c71759-4bc8-4349-9cb1-3cc4730ddf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.238365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b8c71759-4bc8-4349-9cb1-3cc4730ddf60 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:22.239574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b8c71759-4bc8-4349-9cb1-3cc4730ddf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:22.242223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:22.242841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.243113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.243378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.244513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:22.245951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.247814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 91/179] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:22.251826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.270020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.282768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.308131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 89/180] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:22.309379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bf061-b14e-45b8-a251-4b4debf4c7e5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.309661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bf061-b14e-45b8-a251-4b4debf4c7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.309934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bf061-b14e-45b8-a251-4b4debf4c7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.328435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a48bf061-b14e-45b8-a251-4b4debf4c7e5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:22.329616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a48bf061-b14e-45b8-a251-4b4debf4c7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:22.363189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e4a9a-9a3e-4f5f-9405-72a52e8cf7ee None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:22.398072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 90/181] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:22.399138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.399422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.399690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.400793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181753-7a5e-4cf9-8a59-4552e2dff0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:22.401417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181753-7a5e-4cf9-8a59-4552e2dff0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.401692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181753-7a5e-4cf9-8a59-4552e2dff0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.402017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181753-7a5e-4cf9-8a59-4552e2dff0f2 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.403216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181753-7a5e-4cf9-8a59-4552e2dff0f2 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:22.408662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181753-7a5e-4cf9-8a59-4552e2dff0f2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.408662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 92/182] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:22.408662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008571-1f1d-41e0-bfc8-0657961b87f7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.408662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008571-1f1d-41e0-bfc8-0657961b87f7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.408662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008571-1f1d-41e0-bfc8-0657961b87f7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.414419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 93/183] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:22.415028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d008571-1f1d-41e0-bfc8-0657961b87f7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.422698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:22.424163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:22.501380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 91/184] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:22.502567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.536911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.539208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.555064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d008571-1f1d-41e0-bfc8-0657961b87f7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:22.598974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:22.599640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.599925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.600212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.600995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3, user.id=b42994e086454bed9832f0d860d67c67) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:22.601380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:22.602832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.615809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 94/185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/role_assignments?user.id=b42994e086454bed9832f0d860d67c67&scope.project.id=0e51f0f7a2604d7fb35d5ab7dc4dc8e3&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:22.616596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.618447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61417840-4387-4350-82c0-a125c0d0a275 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:22.658778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:22.659748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.660241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.660793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.661722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] RBAC: Adding query filter params (role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=b42994e086454bed9832f0d860d67c67) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:22.662241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:22.663979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f249ce-9f87-45dc-b953-36ae34551a0a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.671627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 92/186] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/role_assignments?user.id=b42994e086454bed9832f0d860d67c67&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:22.677413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.677931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.678397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.716155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:22.717461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:22.833784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 93/187] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:22.834579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.841242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc87255-3362-4ce5-a6ce-6d0268e160ab None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:22.873311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.887733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:22.888382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:22.888818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:22.889094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:22.889905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=f292716c40bd49538840171d62e79f12) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:22.890390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:22.892224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:22.900800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 95/188] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/role_assignments?user.id=f292716c40bd49538840171d62e79f12&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:22.901952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:22.924000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:22.967457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f6a2b-76ad-4844-b127-e39151d4a841 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.012019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.012705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.012987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.013258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.037345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:23.042278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:23.043172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:23.043954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 94/189] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:23.045447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.073975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ec991-d39a-4b27-b9f0-e3f8f4994a51 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.127716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.128309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.128763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.129039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.142064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:23.147604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:23.148727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:23.149537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:23.150702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 96/190] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:23.151490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.178699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.192836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0eb48-35b5-47b1-8349-e316b5de1d8e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.237540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.238175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.238455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.238873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.248442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.249496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:23.254915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:23.255806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:23.256629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 95/191] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:23.257295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.275992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6906f8-bc1f-412f-8880-74c44fea2603 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.309627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.310231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.310512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.310782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.311434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:23.311853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.313321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.321160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 97/192] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.321938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.361069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0e137-3cbd-4df4-a06c-a0cffae06429 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.395631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.396165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.396469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.396741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.397410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:23.397713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.399091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.407740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 96/193] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.408552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.423320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.425240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1c470-cd2c-4c02-bd52-c6c845b5bbb0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.458118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87071324-a106-4e31-8079-548655605fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.458743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87071324-a106-4e31-8079-548655605fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.459023 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87071324-a106-4e31-8079-548655605fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.459292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87071324-a106-4e31-8079-548655605fc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.460416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87071324-a106-4e31-8079-548655605fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.461849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87071324-a106-4e31-8079-548655605fc0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.463399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 98/194] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.464150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87071324-a106-4e31-8079-548655605fc0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.468981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.510121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7a80-e0db-481c-b2fb-6af917b4ec7d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.543801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.544435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.544742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.544990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.545652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:23.545955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.548000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.555727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 97/195] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.556541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.568836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87071324-a106-4e31-8079-548655605fc0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.601955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.602574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.602863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.603115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] PATH_INFO: `/v3/users/9646aa77095b4c6f88225c2256085224` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.616809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=9646aa77095b4c6f88225c2256085224)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.618262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.620172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 99/196] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/users/9646aa77095b4c6f88225c2256085224 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.620948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.633371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.659158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5020f31-dbfa-4e0a-bdde-9eb808945df9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.681040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.692951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.693554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.693830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.694098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] PATH_INFO: `/v3/users/15ff1b46ca5c46c0bc07b4034bf5f512` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.695212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] RBAC: Authorizing `identity:get_user(user_id=15ff1b46ca5c46c0bc07b4034bf5f512)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.696698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.698534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 98/197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/users/15ff1b46ca5c46c0bc07b4034bf5f512 => generated 289 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.699363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.722636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ee2e8-fcbe-4582-ba91-2696e2b9c7f3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.755893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.756518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.756797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.757069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.757691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.759112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.786583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 100/198] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:23 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:23.786583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.801514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb801158-6f5c-42e1-94b1-f5cec8986961 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.844315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.844967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.845247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.845520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] PATH_INFO: `/v3/users/9646aa77095b4c6f88225c2256085224` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.847268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] RBAC: Authorizing `identity:get_user(user_id=9646aa77095b4c6f88225c2256085224)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:23.848807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:23.851442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 99/199] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/users/9646aa77095b4c6f88225c2256085224 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:23.852453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:23.869150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.923124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b96b27-f807-4e43-ace8-32f69a14c87b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:23.942529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:23.967417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:23.968628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:23.968918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:23.969195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:23.978351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c18fa-b5af-482d-826c-9e3671a7a452 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.022512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.023125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.023405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.023677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.059418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.061142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.115924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.118156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.131684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.179197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 101/200] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:24.180146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.199960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.247239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 100/201] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:24.248288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.318114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63272-a2ed-4986-b2bd-67b19ec22a03 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.363168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4613-fea4-4803-9dfb-b07aa8d303a4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.365539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.366228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.366584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.366930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.376999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.409839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.410479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.410756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.411025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.412643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.414768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.416737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 101/202] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:24.418492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.449071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.451394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.476179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.539868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcaad1-5927-4d5c-a938-21c4e225b202 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.580223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 102/203] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:24.581337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.583372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.584111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.584416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.584767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.585964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.587511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.589436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 102/204] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:24.590202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.646820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.689987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8aaa1c-9389-481a-b67d-5b63374c9128 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.706155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72902cef-22e4-4444-bab7-8190f3c73a4d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.707928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.726951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.727555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.727834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.728106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.728884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=9646aa77095b4c6f88225c2256085224) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:24.729192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.730637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.737354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 103/205] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/role_assignments?user.id=9646aa77095b4c6f88225c2256085224&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:24.738226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.744068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.744743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.745020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.745292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.746454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.748462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.750907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 103/206] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:24.757097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:24.853478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918b8f4-792d-4709-bc8b-711cdd2ba306 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.877196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11166ab-4197-4d11-a38c-692c036ad3a9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:24.891491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.892474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.892916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.893351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.894245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] RBAC: Adding query filter params (role.id=334460cecea042219ed608f1b5ff9096, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3, user.id=15ff1b46ca5c46c0bc07b4034bf5f512) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:24.894704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.894816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:24.896897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5aad5-49dc-4f38-8fa1-c6ef2bb3938e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.903822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 104/207] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/role_assignments?user.id=15ff1b46ca5c46c0bc07b4034bf5f512&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=334460cecea042219ed608f1b5ff9096 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:24.931659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:24.932413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:24.932793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:24.933162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:24.934014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] RBAC: Adding query filter params (role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=9646aa77095b4c6f88225c2256085224) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:24.934415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:24.936002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4714c-d983-4b74-87e3-8b6d88d2ff3f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:24.943118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 104/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:24 2022] GET /identity/v3/role_assignments?user.id=9646aa77095b4c6f88225c2256085224&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:25.958818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6e9e-98a1-419c-b22f-12ed3af266e1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:25.959866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6e9e-98a1-419c-b22f-12ed3af266e1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:25.960401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6e9e-98a1-419c-b22f-12ed3af266e1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:25.961936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 105/209] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:25.967278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:25.968820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:25.969335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:25.989082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:25.990599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:26.010374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50198553-48db-42f1-a5a0-e3838f423535 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.010374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50198553-48db-42f1-a5a0-e3838f423535 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.010374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50198553-48db-42f1-a5a0-e3838f423535 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.010374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 106/210] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:26.021007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.021633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.022105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.045010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:26.047251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:26.069756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 105/211] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:26.109244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:26.120753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 107/212] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:26.351388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:26.388818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:26.461954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440c922-d9ee-45fe-972f-f4e200febfe5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:26.473994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:26.504885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d21d7b-fde3-419a-8441-6929d9b01612 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:26.508367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:26.509297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.509795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.510268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.511670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:26.513472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:26.515453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 106/213] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:26.521132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:26.542152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:26.543182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.543667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.544116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.556407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:26.561924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:26.562817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:26.563594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:26.565977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 108/214] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:26.574769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:26.615203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:26.639005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d135-0146-49ff-ac66-c69a36b52451 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:26.684325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:26.684946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.685227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.685499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.686442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:26.689305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:26.698048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836f8c-a51b-4372-ba3b-3fb0cce76334 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:26.718366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 107/215] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:26 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:02:26.750573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:26.751485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.751910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.752385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.753306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:26.753763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:26.755471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55751195-8fbf-456f-8c1d-ab552d0480d4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:26.766262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 109/216] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:26.772257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:26.814961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:26.894859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58190cc1-9387-445c-bad9-b9678ba1653c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:26.934426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:26.935004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f3b3-3525-48be-b5ec-c222c87bb451 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.935137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.935285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f3b3-3525-48be-b5ec-c222c87bb451 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.935360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.935586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] PATH_INFO: `/v3/users/074f997cd80741bfa41017e3dbe5069e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.935691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f3b3-3525-48be-b5ec-c222c87bb451 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.938448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 110/217] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:26.948781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372eb2-b1b0-4dae-bc0d-a1e7ca7c6ec6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:26.949064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372eb2-b1b0-4dae-bc0d-a1e7ca7c6ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:26.949337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372eb2-b1b0-4dae-bc0d-a1e7ca7c6ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:26.953128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] RBAC: Authorizing `identity:get_user(user_id=074f997cd80741bfa41017e3dbe5069e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:26.954822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:26.957426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 108/218] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:26 2022] GET /identity/v3/users/074f997cd80741bfa41017e3dbe5069e => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:26.964617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:26.977509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-47372eb2-b1b0-4dae-bc0d-a1e7ca7c6ec6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:26.979774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-47372eb2-b1b0-4dae-bc0d-a1e7ca7c6ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:27.084591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 111/219] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:27.086600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0d7c-1d3f-41ad-a246-c61d9eb5896c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.087110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0d7c-1d3f-41ad-a246-c61d9eb5896c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.087544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0d7c-1d3f-41ad-a246-c61d9eb5896c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.089945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 112/220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:27.095263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.095795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.096326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.100131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:27.114702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dcf01-33cd-4eda-831f-87982aaede7e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:27.126978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:27.129059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:27.157891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:27.158818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.159306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.159781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.175529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:27.267457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 113/221] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:27.270211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:27.271680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:27.373002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:27.385582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 109/222] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:02:27.392395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:27.478615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:27.587521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d13cd-cad2-43f1-a4a9-22405e9fa535 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:27.651227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b4ac3-46e3-43a7-97e4-eb9ad494cfa9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:27.652077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:27.652745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.654732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.654732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.674757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:27.681963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:27.682893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:27.683681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:27.688312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 110/223] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:27.688312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:27.700088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:27.701028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.701553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.701999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.703350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:27.705162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:27.707072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 114/224] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:27 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:27.708067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:27.745072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:27.750261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:27.811590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ddff4-c58d-4c1d-9861-e6e880316565 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:27.829894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d690b0d-27ed-46da-9cfe-5f7eaf585f20 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:27.848863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:27.849492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.849779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.850037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.862088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:27.867552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:27.868640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:27.869523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:27.872263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 111/225] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:27.873191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:27.874386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:27.875053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:27.875328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:27.875597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:27.876754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:27.879630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:27.882262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:27.893248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 115/226] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:27.894430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.008907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:28.017551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f76876-394e-49a4-9fe5-595d0e793f3c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.022202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e354192-e0de-4fc5-812e-4750b9feca09 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.033127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:28.058937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.059601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.059885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.060154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.060846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:28.061151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.061840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.062667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.062780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.063183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.063584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.064577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=074f997cd80741bfa41017e3dbe5069e) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:28.065231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.066799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.071273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 116/227] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:28.072258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.074102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 112/228] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:27 2022] GET /identity/v3/role_assignments?user.id=074f997cd80741bfa41017e3dbe5069e&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:28.079047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.184341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa49ea8-fb47-4483-b7d1-6a9e6b91bdc0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.197958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e665aae-7f11-4eb1-bba2-a45608bded4a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.223169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.224076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.224524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.224960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] PATH_INFO: `/v3/users/9646aa77095b4c6f88225c2256085224` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.226291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] RBAC: Authorizing `identity:get_user(user_id=9646aa77095b4c6f88225c2256085224)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.227927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.229721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 117/229] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:28 2022] GET /identity/v3/users/9646aa77095b4c6f88225c2256085224 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:28.236398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.240042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.241230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.241230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.241422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] PATH_INFO: `/v3/users/9646aa77095b4c6f88225c2256085224` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.242446 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] RBAC: Authorizing `identity:get_user(user_id=9646aa77095b4c6f88225c2256085224)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.243883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.245970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 113/230] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:28 2022] GET /identity/v3/users/9646aa77095b4c6f88225c2256085224 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:28.252889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.259969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:28.304427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:28.375054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf4af7-bb11-4380-b39d-e9d3c8a2967b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.404144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b1cd0-de81-4d07-9131-c0e16395bd9b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.416968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.418018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.418490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.419062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.422606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.424564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.427045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 114/231] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:28 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:28.439655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.468374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.469384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.469869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.470331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.471633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.473357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.475528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 118/232] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:28 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:28.481916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:28.625112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:28.641129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b61d64-928f-4e81-baae-5f3514a67765 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.646612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676a40-274b-4363-8202-3dc2e9347b04 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:28.673468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:28.711855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2517af-9192-4b78-8c5e-71fc6341e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.712600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2517af-9192-4b78-8c5e-71fc6341e0c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.713038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2517af-9192-4b78-8c5e-71fc6341e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.713455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2517af-9192-4b78-8c5e-71fc6341e0c3 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/9646aa77095b4c6f88225c2256085224/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.715407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2517af-9192-4b78-8c5e-71fc6341e0c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=28bb0208f1ed49099fb0dd0190df3d6b, user_id=9646aa77095b4c6f88225c2256085224, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.716899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2517af-9192-4b78-8c5e-71fc6341e0c3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.724939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39693a30-7e4c-4245-86ff-a25cee48cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:28.725622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693a30-7e4c-4245-86ff-a25cee48cfc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:28.725899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693a30-7e4c-4245-86ff-a25cee48cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:28.726179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693a30-7e4c-4245-86ff-a25cee48cfc3 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/9646aa77095b4c6f88225c2256085224/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:28.732392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 119/233] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:28 2022] PUT /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/9646aa77095b4c6f88225c2256085224/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:28.734353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39693a30-7e4c-4245-86ff-a25cee48cfc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=28bb0208f1ed49099fb0dd0190df3d6b, user_id=9646aa77095b4c6f88225c2256085224, role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:28.735848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39693a30-7e4c-4245-86ff-a25cee48cfc3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:28.767960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 115/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:28 2022] PUT /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/9646aa77095b4c6f88225c2256085224/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:02:29.278926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5325d-f64a-46d1-bea5-73b04088ee70 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:29.279417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5325d-f64a-46d1-bea5-73b04088ee70 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:29.279417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5325d-f64a-46d1-bea5-73b04088ee70 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:29.281236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 120/235] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:29.286443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b675f-49f2-4d56-91db-2cb093e952b1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:29.286876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b675f-49f2-4d56-91db-2cb093e952b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:29.287060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b675f-49f2-4d56-91db-2cb093e952b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:29.308274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cc3b675f-49f2-4d56-91db-2cb093e952b1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:29.309475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cc3b675f-49f2-4d56-91db-2cb093e952b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:29.387692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 116/236] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:29.417472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e7c4b-5df8-45af-b213-4489a4627f20 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:29.417811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e7c4b-5df8-45af-b213-4489a4627f20 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:29.418094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e7c4b-5df8-45af-b213-4489a4627f20 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:29.419614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 121/237] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:29.425696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:29.425992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:29.426272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:29.448577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:29.449781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:29.543233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 117/238] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:29.743081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e7c4b-5df8-45af-b213-4489a4627f20 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:29.860710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e7c4b-5df8-45af-b213-4489a4627f20 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:29.883419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:29.917100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:29.917744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:29.918025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:29.918296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:29.929389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:29.934582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:29.935441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:29.936882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 122/239] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:29.941599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:29.945964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:29.953488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:30.047062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08ab79-4dbd-4170-84a5-fe44be6bcd80 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:30.082449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:30.083302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.083806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.084252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.085096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:30.085617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:30.087152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:30.088230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c21b5-e60d-4f71-bfaf-6e5b37947551 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:30.096809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 123/240] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:30.109716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:30.168889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:30.169532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.170006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.170123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.171934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:30.173302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:30.175193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 118/241] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:30.180574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:30.196574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:30.252514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3af869-a9ef-49ea-92cb-91c0f95430cb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:30.293625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:30.294953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.295538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.295979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] PATH_INFO: `/v3/users/8729bfb3a23b481db292f3ffc81009b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.311054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] RBAC: Authorizing `identity:get_user(user_id=8729bfb3a23b481db292f3ffc81009b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:30.313544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:30.315842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 124/242] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:30 2022] GET /identity/v3/users/8729bfb3a23b481db292f3ffc81009b3 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:30.321663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:30.339709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:30.348041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f8baf-e84d-45c5-96b4-a482a258fab8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:30.396213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152fd11-8c5a-44a0-b962-81a03e421cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:30.396213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152fd11-8c5a-44a0-b962-81a03e421cfd None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.396213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152fd11-8c5a-44a0-b962-81a03e421cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.396213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152fd11-8c5a-44a0-b962-81a03e421cfd None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.396610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152fd11-8c5a-44a0-b962-81a03e421cfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:30.396610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152fd11-8c5a-44a0-b962-81a03e421cfd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:30.429373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 119/243] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:30 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:30.497919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc51ff-c34d-44c1-ba86-129fca3b7a18 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:30.552599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:30.555865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:30.556838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.558674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.558674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.632816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:30.634641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:30.689992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7c1a-e3f8-4bd9-940c-cff38d5d4a05 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.690242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7c1a-e3f8-4bd9-940c-cff38d5d4a05 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.690473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7c1a-e3f8-4bd9-940c-cff38d5d4a05 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.691951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 120/244] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:30.696942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.697474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.697967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.720612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:30.723347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:30.761788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 125/245] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:30.768644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:30.809684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:30.837644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 121/246] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:30.911127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440485-3de6-4eab-bcd9-0fac15ad44cb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:30.950059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:30.950623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:30.950899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:30.951171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:30.952945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:30.954502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:30.956126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 126/247] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:30 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:30.962136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.066728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:31.085534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8b434-08aa-4eae-93a3-3b0aa8af3d6f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:31.126328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:31.126974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.127258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.127538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.128344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] RBAC: Adding query filter params (role.id=334460cecea042219ed608f1b5ff9096, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=8729bfb3a23b481db292f3ffc81009b3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:31.128658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:31.130120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892d10-63af-4991-aea9-3f8a2dbe0788 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:31.136475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 122/248] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:30 2022] GET /identity/v3/role_assignments?user.id=8729bfb3a23b481db292f3ffc81009b3&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=334460cecea042219ed608f1b5ff9096 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:31.145367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.216974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca80ee9-d707-4816-ba2e-bf0accce0d23 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.217258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca80ee9-d707-4816-ba2e-bf0accce0d23 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.217457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca80ee9-d707-4816-ba2e-bf0accce0d23 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.219023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 123/249] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:31.226089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26546d-aab7-4dca-b5c6-ef911480a499 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.226376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26546d-aab7-4dca-b5c6-ef911480a499 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.226658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26546d-aab7-4dca-b5c6-ef911480a499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.265971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8a26546d-aab7-4dca-b5c6-ef911480a499 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:31.267200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a26546d-aab7-4dca-b5c6-ef911480a499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:31.300389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:31.312128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7cd36-61d2-48af-a92a-e302ebeb44f0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:31.355283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd218ae-e8d3-466e-8904-395cd9b9cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:31.356023 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd218ae-e8d3-466e-8904-395cd9b9cffd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.356330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd218ae-e8d3-466e-8904-395cd9b9cffd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.356602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd218ae-e8d3-466e-8904-395cd9b9cffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.368970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd218ae-e8d3-466e-8904-395cd9b9cffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:31.369304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 124/250] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:31.371090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3c9f-76ca-4933-b452-90a34ab6a24f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.371545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3c9f-76ca-4933-b452-90a34ab6a24f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.372309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3c9f-76ca-4933-b452-90a34ab6a24f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.373984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 125/251] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4bd218ae-e8d3-466e-8904-395cd9b9cffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:31.377395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:31.378243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:31.379018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:31.380725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 127/252] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:31.382056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.382349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.382629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.386473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e3c9f-76ca-4933-b452-90a34ab6a24f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.402811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:31.404098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:31.479693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 128/253] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:31.512023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:31.545883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e3c9f-76ca-4933-b452-90a34ab6a24f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:31.591306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:31.592215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.592667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.593691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.594539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:31.595058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:31.596615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:31.608944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 126/254] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:31.614649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.707734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:31.724263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24289fd-1a29-4a78-91e1-b3b84ea3ca58 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:31.771717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:31.773203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.773771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.774207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] PATH_INFO: `/v3/users/074f997cd80741bfa41017e3dbe5069e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.775521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] RBAC: Authorizing `identity:get_user(user_id=074f997cd80741bfa41017e3dbe5069e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:31.777193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:31.778948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 129/255] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:31 2022] GET /identity/v3/users/074f997cd80741bfa41017e3dbe5069e => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:31.785003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.808526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.937015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c4d5-d514-4749-85f8-b2f1bc5cbdf4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:31.937263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2b503-9c61-4c8d-a744-ce59167ee5a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:31.971915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:31.972756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.973129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.973403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.973641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:31.974273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:31.974518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:31.974610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:31.974807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:31.976154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:31.977916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 127/256] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:31 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:31.978812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:31.991104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:31.995879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:31.997106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:31.997902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 130/257] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:31.998579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.004534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:32.027699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:32.147657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d1300-c4ce-40ca-bd44-fa24bb380ee7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.174160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc698-e5ec-4a36-bead-fcb9f46d6756 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.182845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.183503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.183783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.184049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/074f997cd80741bfa41017e3dbe5069e/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:32.185986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=28bb0208f1ed49099fb0dd0190df3d6b, user_id=074f997cd80741bfa41017e3dbe5069e, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:32.188028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:32.204451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 131/258] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:31 2022] PUT /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/074f997cd80741bfa41017e3dbe5069e/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:32.205342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.238209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.238963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.239246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.239512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:32.257352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:32.262995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:32.263867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:32.264769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 128/259] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:32.279363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.346121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:32.377506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392566-aa3e-4d4c-bca9-b8c097ab55d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.412059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:32.423700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.424715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.429443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.429926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:32.430834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:32.431291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:32.432898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:32.447453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 132/260] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:32.455170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.494588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622e4b-6f76-4148-8539-09a744fd7e16 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.572218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12576-1365-4c1c-b0ec-e22da31e37be None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.601336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.602134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.602636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.602880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:32.603651 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:32.604055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:32.605560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:32.614499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.615351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.615764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.616161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] PATH_INFO: `/v3/users/9646aa77095b4c6f88225c2256085224` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:32.617503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] RBAC: Authorizing `identity:get_user(user_id=9646aa77095b4c6f88225c2256085224)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:32.619091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:32.621149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 133/261] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:32 2022] GET /identity/v3/users/9646aa77095b4c6f88225c2256085224 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:32.624104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 129/262] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:32.627389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.640563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:32.643784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.819760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a5ff3-82a8-40d1-9920-d184eac211f9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.837461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb75e-bd90-4e73-9fee-e96f57a30ce3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:32.883938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:32.890987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.891629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.891910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.892207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] PATH_INFO: `/v3/users/9646aa77095b4c6f88225c2256085224` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:32.893604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] RBAC: Authorizing `identity:get_user(user_id=9646aa77095b4c6f88225c2256085224)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:32.895439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:32.897514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 130/263] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:32 2022] GET /identity/v3/users/9646aa77095b4c6f88225c2256085224 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:32.903188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:32.909844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:32.910475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:32.910758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:32.911031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.067681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:33.069192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:33.076615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:33.126639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8500c-5faa-4717-b7a2-990ba0f90f34 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:33.209502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:33.210134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.210418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.210697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.250622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 134/264] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:33.256397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:33.337833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:33.342204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:33.344302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:33.478141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ed552-4e60-4a87-9b1b-312d816d779e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:33.480577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:33.507170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 131/265] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:02:33.513820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:33.559282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:33.560336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.560832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.561361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.568886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:33.570559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:33.572803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 135/266] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:33 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:33.579250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:33.645874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:33.651382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df6dc-f78f-4e1a-93b4-cff6846f9b3c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:33.731434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ad986-d64a-4e04-8e42-7421df962e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:33.731990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ad986-d64a-4e04-8e42-7421df962e24 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.732296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ad986-d64a-4e04-8e42-7421df962e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.732573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ad986-d64a-4e04-8e42-7421df962e24 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.733960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ad986-d64a-4e04-8e42-7421df962e24 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:33.735438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ad986-d64a-4e04-8e42-7421df962e24 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:33.741342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 132/267] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:33 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:33.743055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29de44d-ccbd-4795-9c52-42c9c25df6d7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.743527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29de44d-ccbd-4795-9c52-42c9c25df6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.743984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29de44d-ccbd-4795-9c52-42c9c25df6d7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.745469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 133/268] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:33.747751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29de44d-ccbd-4795-9c52-42c9c25df6d7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:33.775985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fda98-1d79-4f3d-9011-e8508ff02ca7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:33.828254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:33.828909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.829187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.829461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.830192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=9646aa77095b4c6f88225c2256085224) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:33.830501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:33.831946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901710c-0144-4715-90a4-ea8aebf9615b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:33.849564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 136/269] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:33 2022] GET /identity/v3/role_assignments?user.id=9646aa77095b4c6f88225c2256085224&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:33.850822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78019e10-4bae-426e-a41c-0e3c3a4c9af1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.851106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78019e10-4bae-426e-a41c-0e3c3a4c9af1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.851382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78019e10-4bae-426e-a41c-0e3c3a4c9af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.873145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-78019e10-4bae-426e-a41c-0e3c3a4c9af1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:33.875390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-78019e10-4bae-426e-a41c-0e3c3a4c9af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:33.912001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:33.992096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 137/270] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:33.993489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f217-01f3-4638-a0fe-a3ad9f21a7a8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:33.993776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f217-01f3-4638-a0fe-a3ad9f21a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:33.994222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f217-01f3-4638-a0fe-a3ad9f21a7a8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:33.995671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 138/271] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:34.003781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.004097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.004412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.005409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29de44d-ccbd-4795-9c52-42c9c25df6d7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.041154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:34.042427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:34.056469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:34.057075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.057352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.057626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.058353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] RBAC: Adding query filter params (role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=9646aa77095b4c6f88225c2256085224) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:34.058659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:34.061102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40495560-3706-48a9-b48f-381b8014a05f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:34.068166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 134/272] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:33 2022] GET /identity/v3/role_assignments?user.id=9646aa77095b4c6f88225c2256085224&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:34.069291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a51b9d-3d8b-4495-984f-a56a66e7f102 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.069616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a51b9d-3d8b-4495-984f-a56a66e7f102 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.070663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a51b9d-3d8b-4495-984f-a56a66e7f102 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.071984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 135/273] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:34.081492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.081797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.082037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.102820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:34.104059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:34.133802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:34.154461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 139/274] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:34.223157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 136/275] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:34.318754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.437442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:34.442763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf276d-c950-4176-a17a-1eea2d2f4c4c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.481934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:34.484642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.484933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.485205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.496027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:34.502083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:34.502968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:34.503754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 140/276] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:34.510305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.536284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.644359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a296-a112-4a05-9b1e-425b86ee75b4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.648571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46544a-7681-4e7b-8035-bc700b742745 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.681438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:34.682045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.682327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.682599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.683266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:34.683555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:34.684974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:34.688089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:34.688950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.689403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.689850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.693429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 137/277] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:34.694042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.703229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:34.708452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:34.709389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:34.710360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:34.712172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 141/278] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:34.713196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.729138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:34.731078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:34.808582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07273b-0eeb-490f-beb9-02f8a364876c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.828914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb0c86-933f-40d7-a12c-76f8e4b00437 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.843945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:34.844605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.844881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.845149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.846259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:34.847700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:34.849293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 138/279] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:34.850046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.873365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:34.874249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:34.874535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:34.874789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] PATH_INFO: `/v3/users/8729bfb3a23b481db292f3ffc81009b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:34.875922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] RBAC: Authorizing `identity:get_user(user_id=8729bfb3a23b481db292f3ffc81009b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:34.877383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:34.878949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 142/280] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:34 2022] GET /identity/v3/users/8729bfb3a23b481db292f3ffc81009b3 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:34.879688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:34.956779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:34.966268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:34.968610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4008a57-ab31-40f1-a36f-64ed1b36c89a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:34.987215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccc2df-a2a5-4df6-ad4b-21afa2e236be None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:35.003049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.003660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.003939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.004254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.004888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:35.005192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:35.006582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:35.015688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 139/281] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:35.016510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:35.025302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1d355-ed05-42d3-b7df-f1ed97c50542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.025924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1d355-ed05-42d3-b7df-f1ed97c50542 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.026202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1d355-ed05-42d3-b7df-f1ed97c50542 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.026472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1d355-ed05-42d3-b7df-f1ed97c50542 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.027489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1d355-ed05-42d3-b7df-f1ed97c50542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:35.029264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1d355-ed05-42d3-b7df-f1ed97c50542 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:35.050186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 143/282] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:02:34 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:35.051407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b025b57-9ba4-425e-abb4-03ce5aa18b3b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.051701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b025b57-9ba4-425e-abb4-03ce5aa18b3b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.051968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b025b57-9ba4-425e-abb4-03ce5aa18b3b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.053384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 144/283] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:35.057230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b025b57-9ba4-425e-abb4-03ce5aa18b3b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:35.129799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7908a7-6ae3-4588-acc0-0421d6c35a06 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:35.169434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa0fec-4c12-405e-834e-889afb300b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.170076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0fec-4c12-405e-834e-889afb300b39 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.170362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0fec-4c12-405e-834e-889afb300b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.170626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0fec-4c12-405e-834e-889afb300b39 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.172614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0fec-4c12-405e-834e-889afb300b39 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:35.174098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0fec-4c12-405e-834e-889afb300b39 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:35.180809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 140/284] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:35 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:35.180809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.180809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.180809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.191301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b025b57-9ba4-425e-abb4-03ce5aa18b3b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:35.198561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:35.199765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:35.215825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:35.237488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.238125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.238406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.238679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] PATH_INFO: `/v3/users/9a7ca247d418414ea4f058e4c922933c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.247550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:35.256750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] RBAC: Authorizing `identity:get_user(user_id=9a7ca247d418414ea4f058e4c922933c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:35.259499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:35.261416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 145/285] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:35 2022] GET /identity/v3/users/9a7ca247d418414ea4f058e4c922933c => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:35.262177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:35.282508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 141/286] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:35.283642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:35.411003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e052aa-7f5a-473f-9e1a-7ad90a090a5f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:35.412570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e3cbc-9095-4050-8fd8-48fd7976e313 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:35.458859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.459473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.459757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.460030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.460802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, user.id=9a7ca247d418414ea4f058e4c922933c, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:35.461109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:35.462617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:35.463343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.464546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.465029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.466004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/8729bfb3a23b481db292f3ffc81009b3/roles/334460cecea042219ed608f1b5ff9096` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.469128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=28bb0208f1ed49099fb0dd0190df3d6b, user_id=8729bfb3a23b481db292f3ffc81009b3, role_id=334460cecea042219ed608f1b5ff9096)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:35.470463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 142/287] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Aug 18 11:02:35 2022] GET /identity/v3/role_assignments?user.id=9a7ca247d418414ea4f058e4c922933c&scope.system=all&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:35.471947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:35.490745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 146/288] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:02:35 2022] PUT /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/users/8729bfb3a23b481db292f3ffc81009b3/roles/334460cecea042219ed608f1b5ff9096 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:35.591085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:35.614366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:35.718401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8d976-b300-43bd-9cf1-fc4a9bfbdb32 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:35.757536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:35.758552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:35.759028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:35.759529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:35.771371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:35.776415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:35.777261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:35.778055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:35.780973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 143/289] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:35.786737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:35.914971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:35.984253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147725d4-93fd-4ed5-9921-b6f8f4596cb0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:36.019095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:36.019698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:36.019970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:36.020256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:36.021011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:36.021310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:36.022737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:36.035557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 147/290] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:36.040674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:36.145929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:36.179332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324912c7-89de-4c1d-8fbd-bc869cbe4265 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:36.229996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:36.230619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:36.230939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:36.231211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] PATH_INFO: `/v3/users/074f997cd80741bfa41017e3dbe5069e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:36.232929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] RBAC: Authorizing `identity:get_user(user_id=074f997cd80741bfa41017e3dbe5069e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:36.234716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:36.236772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 144/291] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:36 2022] GET /identity/v3/users/074f997cd80741bfa41017e3dbe5069e => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:36.242560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:36.392105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab291837-37db-4d7f-8a2e-4d00ed4d696c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:36.413982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:36.438349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:36.438967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:36.439244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:36.439516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:36.522017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:36.523403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:36.632326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 148/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:36.637466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:36.797609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6e838-2269-4bdd-9b86-c5a92a0a1477 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:36.848630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8882011-b698-4212-90e4-a1f4df3e578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:36.849655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882011-b698-4212-90e4-a1f4df3e578e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:36.850169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882011-b698-4212-90e4-a1f4df3e578e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:36.850636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882011-b698-4212-90e4-a1f4df3e578e None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:36.856868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882011-b698-4212-90e4-a1f4df3e578e None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:36.858584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882011-b698-4212-90e4-a1f4df3e578e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:36.860469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 145/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:36 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:36.866281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:36.900735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:36.999367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2fb51-a77e-4643-bdf8-85d0957bec29 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:37.052644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:37.053185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.053466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.053737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.054469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=074f997cd80741bfa41017e3dbe5069e) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:37.054773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:37.056217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98eeb-d833-46c0-bdbd-bcdc77c1ce9c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:37.065064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 149/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:36 2022] GET /identity/v3/role_assignments?user.id=074f997cd80741bfa41017e3dbe5069e&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:37.696622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08d017-cc3b-4bab-addc-4d1ca14b597f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.697154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08d017-cc3b-4bab-addc-4d1ca14b597f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.697468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08d017-cc3b-4bab-addc-4d1ca14b597f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.699043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 146/295] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:37.705061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1e4e8-8e9c-49d9-9f0a-56c00a71cad7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.705520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1e4e8-8e9c-49d9-9f0a-56c00a71cad7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.705597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1e4e8-8e9c-49d9-9f0a-56c00a71cad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.727232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f8d1e4e8-8e9c-49d9-9f0a-56c00a71cad7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:37.728968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f8d1e4e8-8e9c-49d9-9f0a-56c00a71cad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:37.729352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d761f9-c5d2-46f6-b55f-2e6e8976c62b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.729635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d761f9-c5d2-46f6-b55f-2e6e8976c62b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.729907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d761f9-c5d2-46f6-b55f-2e6e8976c62b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.731234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 147/296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:37.736648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ad4d3-9fe2-489d-9931-d9d5c2a25c7f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.736927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ad4d3-9fe2-489d-9931-d9d5c2a25c7f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.737198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ad4d3-9fe2-489d-9931-d9d5c2a25c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.759974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3b6ad4d3-9fe2-489d-9931-d9d5c2a25c7f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:37.761411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3b6ad4d3-9fe2-489d-9931-d9d5c2a25c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:37.765112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:37.813075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 150/297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:37.814992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2c72-3d9c-4de8-944f-355f4cc586dd None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.815694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2c72-3d9c-4de8-944f-355f4cc586dd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.816106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2c72-3d9c-4de8-944f-355f4cc586dd None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.818350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 151/298] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:37.823712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.823994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.824289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.843791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 148/299] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:37.851356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:37.852801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:37.854741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8551-5840-470d-ade5-6db06eba7eab None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.855027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8551-5840-470d-ade5-6db06eba7eab None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.855306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8551-5840-470d-ade5-6db06eba7eab None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.856942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 149/300] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:37.862630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:37.862914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:37.863190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:37.863942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:37.885347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:37.886528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:37.941220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 152/301] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:37.968535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 150/302] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:38.085869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.127094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.195365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdf15d-9006-4188-83fb-5eee8319b3af None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.205550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.237596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.238481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.238760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.239034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.241562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eede822-a0a5-40f9-8cbc-1f0816e642d8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.250113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:38.255036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:38.255893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:38.256737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 153/303] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:38.258140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.261238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.283363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.283997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.284310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.284587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.294711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:38.299954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:38.301044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 151/304] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:38.302311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.365209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1491c-33c5-4064-8c3c-0782d6e7df05 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.399530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.400150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.400461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.400733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.401849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.403276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.403438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086ad8-3d7c-417f-8141-bae114a0384f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.404861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 154/305] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:38.405770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.434061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.440940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.441540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.441817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.442077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.442729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:38.443025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.444416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.453421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 152/306] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:38.454194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.474444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.507491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1eb95-7de4-42c5-8095-d477c95219fc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.545537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.546189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.546472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.546747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.556633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:38.558008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677bdae-fc30-41d8-a3ed-5f786cd33e8f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:38.562492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:38.563364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:38.564167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 155/307] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:38.564845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.593190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.593788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.594063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.594331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.594989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:38.595291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.596702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.604664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 153/308] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:38.605442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.661076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.666547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312e8f-6ad9-4bb3-a441-049b2491bec5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.695291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.701230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.702434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.702725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.702997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] PATH_INFO: `/v3/groups` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.703618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.705064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.707123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83afdb-7d26-47c4-a2ec-40b73936359b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.721057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 156/309] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Aug 18 11:02:38 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:02:38.728168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.744116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.744823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.745102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.745384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] PATH_INFO: `/v3/users/9a7ca247d418414ea4f058e4c922933c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.746602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=9a7ca247d418414ea4f058e4c922933c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.748388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.755384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 154/310] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/users/9a7ca247d418414ea4f058e4c922933c => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:38.755384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.852864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f2531-4c7f-4f8e-babc-5c2fb3e59161 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.869278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e57c0-9f30-40e7-8c96-ac4b329d1ad3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:38.893626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.893943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.894159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.895957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.896696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:38.897002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.899542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:38.900121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.909911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 157/311] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:38.911217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.914421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:38.915059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:38.915332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:38.915598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] PATH_INFO: `/v3/users/5a5b4cb008f24bba8f208ee46041a185` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:38.929775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5b4cb008f24bba8f208ee46041a185)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:38.931213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:38.933071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 155/312] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/users/5a5b4cb008f24bba8f208ee46041a185 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:38.933640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:38.965861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:39.018548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b53409-cb1f-4191-a955-574950b43204 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:39.052750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88885e2-afab-40e9-9c11-356fae5df4b3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:39.057647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:39.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:39.058974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:39.059423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] PATH_INFO: `/v3/system/users/9a7ca247d418414ea4f058e4c922933c/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:39.061235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9a7ca247d418414ea4f058e4c922933c, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:39.062931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:39.073798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 158/313] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Aug 18 11:02:38 2022] PUT /identity/v3/system/users/9a7ca247d418414ea4f058e4c922933c/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:39.075068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:39.100444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:39.101440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:39.101900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:39.102350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] PATH_INFO: `/v3/users/8729bfb3a23b481db292f3ffc81009b3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:39.104071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] RBAC: Authorizing `identity:get_user(user_id=8729bfb3a23b481db292f3ffc81009b3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:39.105820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:39.107937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 156/314] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:38 2022] GET /identity/v3/users/8729bfb3a23b481db292f3ffc81009b3 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:39.114204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:39.197394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:39.218348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c2b4-a699-4c7d-a127-957fb6773c36 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:39.240286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:39.273888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dde997-e62a-481c-a95e-3232f8a60f09 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:39.275716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:39.277461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:39.277895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:39.278354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:39.279248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] RBAC: Adding query filter params (role.id=334460cecea042219ed608f1b5ff9096, user.id=5a5b4cb008f24bba8f208ee46041a185, scope.system=all) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:39.279701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:39.281415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:39.288719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 159/315] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Aug 18 11:02:39 2022] GET /identity/v3/role_assignments?user.id=5a5b4cb008f24bba8f208ee46041a185&scope.system=all&role.id=334460cecea042219ed608f1b5ff9096 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:39.320424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:39.321228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:39.321606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:39.321982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:39.405810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:39.407209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:39.462482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:39.517353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 157/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:02:39.522759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:39.638735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50c86-93b0-47f8-8d83-f8a8ebf6e463 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:39.676555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aecfb0-b26a-41f4-9e16-bdf25132c443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:39.677188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aecfb0-b26a-41f4-9e16-bdf25132c443 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:39.677469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aecfb0-b26a-41f4-9e16-bdf25132c443 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:39.677745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aecfb0-b26a-41f4-9e16-bdf25132c443 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:39.678877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aecfb0-b26a-41f4-9e16-bdf25132c443 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:39.680362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aecfb0-b26a-41f4-9e16-bdf25132c443 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:39.685958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 160/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:39 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:39.689481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:39.796654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794fca8-a60c-4f37-a1bc-1e94ceb17d72 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:39.831983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:39.832610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:39.832887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:39.833156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:39.833871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] RBAC: Adding query filter params (role.id=334460cecea042219ed608f1b5ff9096, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b, user.id=8729bfb3a23b481db292f3ffc81009b3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:39.834170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:39.835610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d652-8b8f-4ed3-8686-aeaa9c814dc9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:39.841947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 158/318] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:02:39 2022] GET /identity/v3/role_assignments?user.id=8729bfb3a23b481db292f3ffc81009b3&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=334460cecea042219ed608f1b5ff9096 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:40.701052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254e8d8-b91f-493d-9cf7-8ff61d7c0169 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:40.701575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254e8d8-b91f-493d-9cf7-8ff61d7c0169 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:40.701575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254e8d8-b91f-493d-9cf7-8ff61d7c0169 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:40.703660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 161/319] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:40.708778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a57e1-e65f-47c2-8261-a73d0a283826 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:40.709068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a57e1-e65f-47c2-8261-a73d0a283826 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:40.709342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a57e1-e65f-47c2-8261-a73d0a283826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:40.727207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af7a57e1-e65f-47c2-8261-a73d0a283826 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:40.728412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af7a57e1-e65f-47c2-8261-a73d0a283826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:40.736728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:40.800272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 159/320] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:41.069527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254e8d8-b91f-493d-9cf7-8ff61d7c0169 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.070570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac277c2-7a1e-468c-97c9-5ed1426f0da4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.070859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac277c2-7a1e-468c-97c9-5ed1426f0da4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.071135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac277c2-7a1e-468c-97c9-5ed1426f0da4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.072931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 160/321] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:41.078947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2ef9-d31b-4dcd-8c4d-74b2af4439df None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.079232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2ef9-d31b-4dcd-8c4d-74b2af4439df None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.079509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2ef9-d31b-4dcd-8c4d-74b2af4439df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.098169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-01bc2ef9-d31b-4dcd-8c4d-74b2af4439df None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:41.099362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01bc2ef9-d31b-4dcd-8c4d-74b2af4439df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:41.136697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:41.168272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 161/322] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:41.198802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254e8d8-b91f-493d-9cf7-8ff61d7c0169 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.225747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cc4fb-8f11-4d4a-94ef-a95cd4aa7f22 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.226030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cc4fb-8f11-4d4a-94ef-a95cd4aa7f22 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.226295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cc4fb-8f11-4d4a-94ef-a95cd4aa7f22 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.227629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 162/323] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:41.232293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.232573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.232840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.234893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.235517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.235793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.236064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.237196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:41.238629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:41.240867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 162/324] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:41.245375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.256758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:41.258292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:41.340254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 163/325] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:41.354400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21db1ff-50a2-48f0-bc45-8c3e9a932481 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.370992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:41.390451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.391066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.391332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.391604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] PATH_INFO: `/v3/groups` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.392234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:41.393670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:41.409732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 163/326] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Aug 18 11:02:41 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:02:41.440612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.443621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:41.555035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc15aa-bde0-44ce-9f20-32e3e9c7832e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.590681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.591243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.591525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.591797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.598124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.604953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:41.610385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:41.611272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:41.612077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 164/327] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:41.617442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.697984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:41.745027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70cd78-5d41-4286-8f62-254798be686b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.749064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969077ad-1acb-4f37-bfa9-640a290d6467 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.783774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.784470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.784738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.785002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.792390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.793277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.793773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.794239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.795122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:41.795650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:41.796988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:41.797600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:41.802369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:41.803459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:41.804494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 164/328] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:41.811061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.812420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 165/329] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:41.818902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.857679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:41.929537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f028a5-c7eb-400f-af7a-43b1820d00fd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.933361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f9be4-a362-41a8-98c1-141fc2161279 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:41.954797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:41.968545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.969263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.969734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.970151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] PATH_INFO: `/v3/users/9a7ca247d418414ea4f058e4c922933c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.971480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] RBAC: Authorizing `identity:get_user(user_id=9a7ca247d418414ea4f058e4c922933c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:41.973416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:41.975576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:41.975884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 166/330] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/users/9a7ca247d418414ea4f058e4c922933c => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:41.976687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:41.977102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:41.977515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:41.978183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:41.978486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:41.979861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:41.983605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:41.991623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 165/331] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:41.997536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:42.093562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd090f6-3fec-4e14-9b6a-80e160015953 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:42.114094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc90682-9ce3-4542-aaa2-1c952f255021 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:42.129472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:42.130383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:42.130890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:42.131377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:42.132349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] RBAC: Adding query filter params (role.id=866bc5028f7b4637aa1ae91761cd03b5, user.id=9a7ca247d418414ea4f058e4c922933c, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:42.132863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:42.134514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:42.147827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 167/332] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/role_assignments?user.id=9a7ca247d418414ea4f058e4c922933c&scope.system=all&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:42.149171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:42.165089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea12e7-e7b3-43ea-8255-1c1b9a4555ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:42.165814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea12e7-e7b3-43ea-8255-1c1b9a4555ee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:42.166102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea12e7-e7b3-43ea-8255-1c1b9a4555ee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:42.166377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea12e7-e7b3-43ea-8255-1c1b9a4555ee None admin] PATH_INFO: `/v3/users/5a5b4cb008f24bba8f208ee46041a185` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:42.167676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea12e7-e7b3-43ea-8255-1c1b9a4555ee None admin] RBAC: Authorizing `identity:get_user(user_id=5a5b4cb008f24bba8f208ee46041a185)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:42.169170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea12e7-e7b3-43ea-8255-1c1b9a4555ee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:42.170879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 166/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:41 2022] GET /identity/v3/users/5a5b4cb008f24bba8f208ee46041a185 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:42.176757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:42.241748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:42.301487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba6ac2-a8c1-4dbd-b10e-965e9370b2f9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:42.351910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c47b5-e935-45b1-a60c-02bb68eb6598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:42.352742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c47b5-e935-45b1-a60c-02bb68eb6598 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:42.353017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c47b5-e935-45b1-a60c-02bb68eb6598 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:42.353288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c47b5-e935-45b1-a60c-02bb68eb6598 None admin] PATH_INFO: `/v3/system/users/5a5b4cb008f24bba8f208ee46041a185/roles/334460cecea042219ed608f1b5ff9096` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:42.354720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c47b5-e935-45b1-a60c-02bb68eb6598 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a5b4cb008f24bba8f208ee46041a185, role_id=334460cecea042219ed608f1b5ff9096)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:42.356106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c47b5-e935-45b1-a60c-02bb68eb6598 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:42.369547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 168/334] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Aug 18 11:02:42 2022] PUT /identity/v3/system/users/5a5b4cb008f24bba8f208ee46041a185/roles/334460cecea042219ed608f1b5ff9096 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:02:43.605754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa565a12-7094-4fcd-8996-ababaa1d2936 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:43.606818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa565a12-7094-4fcd-8996-ababaa1d2936 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:43.607368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa565a12-7094-4fcd-8996-ababaa1d2936 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:43.610029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 167/335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:43.616327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9797ca-3470-4296-9ec7-50e463dedaef None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:43.617157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9797ca-3470-4296-9ec7-50e463dedaef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:43.617693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9797ca-3470-4296-9ec7-50e463dedaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:43.640450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e9797ca-3470-4296-9ec7-50e463dedaef None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:43.641976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e9797ca-3470-4296-9ec7-50e463dedaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:43.720580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 169/336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:44.110644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669aeae-1943-495b-8a2d-9bb40ff8caa7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.110990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669aeae-1943-495b-8a2d-9bb40ff8caa7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.111205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669aeae-1943-495b-8a2d-9bb40ff8caa7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.112903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 168/337] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:44.118680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b414f2d-a322-41ab-9590-61c377c975ba None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.118973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b414f2d-a322-41ab-9590-61c377c975ba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.119252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b414f2d-a322-41ab-9590-61c377c975ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.141160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6b414f2d-a322-41ab-9590-61c377c975ba None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:44.142415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6b414f2d-a322-41ab-9590-61c377c975ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:44.194043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:44.228613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 170/338] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:44.364449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669aeae-1943-495b-8a2d-9bb40ff8caa7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:44.385394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ec0e-b3e4-42cc-a9fe-2547e3a653d9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.386024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ec0e-b3e4-42cc-a9fe-2547e3a653d9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.386516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ec0e-b3e4-42cc-a9fe-2547e3a653d9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.388343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 171/339] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:44.394427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.394987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.395457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.427648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:44.429111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:44.531825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669aeae-1943-495b-8a2d-9bb40ff8caa7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:44.558733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 172/340] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:44.587597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:44.588264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.588546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.588823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.608671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:44.613459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:44.614322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:44.615143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 169/341] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:44.622428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:44.660226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:44.711942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:44.792978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315a2fc-2b37-4b6c-ab35-6ea1abeebe55 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:44.795859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:44.861085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:44.862181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.863441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.864562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.865652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:44.867139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:44.868937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:44.878754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f66b7-fe21-46fe-9e70-5330e671fa27 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:44.889646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 173/342] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:44.891468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:44.951020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:44.952755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:44.954906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:44.955481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:44.983207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:44.988141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:44.989067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:44.989868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:44.996385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 170/343] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:44.997691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.092033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461ad3a-2cd5-4f76-b2f2-37b843d71adf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.127917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.135435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96273c-1d07-4076-81aa-a0c6a81d16d4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.168940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.169569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.169851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.170131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.173628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.174258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.174534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.174807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.186759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:45.191858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:45.192773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:45.193569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:45.196031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 171/344] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:45.197098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.243595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.260422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:45.262187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:45.319271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8fc50-666d-46c3-b2c7-d41e468977b8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.355967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.357026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.357534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.358119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.359466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:45.360276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:45.363054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:45.377043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 172/345] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:45.378187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.396598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.425206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 174/346] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:45.426193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.489283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2605e-6804-4c4d-ae48-5a1001bdfcd1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.529611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.530592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.531087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.531663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.532654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:45.533246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:45.534815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:45.536587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.547265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 173/347] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:45.548654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.588490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc257-6348-461a-a5a4-2987ae0aecb0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.626131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e9758-7e51-42d7-9afe-2bdc5e2a06f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.626756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9758-7e51-42d7-9afe-2bdc5e2a06f5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.627000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9758-7e51-42d7-9afe-2bdc5e2a06f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.627232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9758-7e51-42d7-9afe-2bdc5e2a06f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.636867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.669746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c382c-95f0-4235-99db-cde582b87c89 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.701159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e9758-7e51-42d7-9afe-2bdc5e2a06f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:45.702983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e9758-7e51-42d7-9afe-2bdc5e2a06f5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:45.714527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d709564-0993-4ff2-828d-ddf993cd2a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.715286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709564-0993-4ff2-828d-ddf993cd2a18 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.715465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709564-0993-4ff2-828d-ddf993cd2a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.715701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709564-0993-4ff2-828d-ddf993cd2a18 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.716841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d709564-0993-4ff2-828d-ddf993cd2a18 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:45.718464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d709564-0993-4ff2-828d-ddf993cd2a18 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:45.720427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 174/348] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:45 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:45.721574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db67ad-4136-4c36-b498-b46621de1745 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.721854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db67ad-4136-4c36-b498-b46621de1745 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.722125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db67ad-4136-4c36-b498-b46621de1745 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.723334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 175/349] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:45.724103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db67ad-4136-4c36-b498-b46621de1745 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.776939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.821677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 175/350] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:45.823076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31600c1-e200-4ebd-9162-8109a862909c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.823429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31600c1-e200-4ebd-9162-8109a862909c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.823769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31600c1-e200-4ebd-9162-8109a862909c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.825387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 176/351] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:45.825881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31600c1-e200-4ebd-9162-8109a862909c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:45.859257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db67ad-4136-4c36-b498-b46621de1745 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:45.910634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46af42-cc03-40fb-9b31-230355aa7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:45.911303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46af42-cc03-40fb-9b31-230355aa7f0c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:45.911578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46af42-cc03-40fb-9b31-230355aa7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:45.911846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46af42-cc03-40fb-9b31-230355aa7f0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:45.923328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:45.969337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31600c1-e200-4ebd-9162-8109a862909c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.011073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46af42-cc03-40fb-9b31-230355aa7f0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.012837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46af42-cc03-40fb-9b31-230355aa7f0c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:46.018604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8fab9-6f8a-41ca-a519-94437c292bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.019248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8fab9-6f8a-41ca-a519-94437c292bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.019524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8fab9-6f8a-41ca-a519-94437c292bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.019792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8fab9-6f8a-41ca-a519-94437c292bd4 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.027095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8fab9-6f8a-41ca-a519-94437c292bd4 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.028628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8fab9-6f8a-41ca-a519-94437c292bd4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:46.031375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 177/352] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:45 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:46.033388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.033388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.033388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.060591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:46.062488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:46.096404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:46.160222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 176/353] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:46.160222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.160222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.160222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.164353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 178/354] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:46.165214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.180496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:46.182184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:46.220273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:46.273340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 177/355] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:46.274766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.294161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c818a-d704-4a24-80fb-c3bb96188d2d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.347722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.348875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.350348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.350855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.353626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.356101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:46.361115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 179/356] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:46.363202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.410089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:46.428207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766898f-7d59-4a3e-b705-0013726a315f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.484694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.485305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.485576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.485837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.486560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:46.486856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.489071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:46.494600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 178/357] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:46.495300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.536679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30dac8-4d35-449d-aed6-6309d2531d02 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.546539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:46.580847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.581517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.581797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.582066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.583242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.584762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:46.586811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 180/358] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:46.587347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.633354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce996-d2fe-4eb1-a867-3da9082a8631 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.678869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8a0af-0b6b-4546-aa01-4b5ee48bd8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.680324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8a0af-0b6b-4546-aa01-4b5ee48bd8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.680809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8a0af-0b6b-4546-aa01-4b5ee48bd8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.681287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8a0af-0b6b-4546-aa01-4b5ee48bd8c0 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.682448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8a0af-0b6b-4546-aa01-4b5ee48bd8c0 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.683889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8a0af-0b6b-4546-aa01-4b5ee48bd8c0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:46.685501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 179/359] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:46.687189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c17e5-f758-4d40-b6a8-2d4cf7486f59 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.687572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c17e5-f758-4d40-b6a8-2d4cf7486f59 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.687850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c17e5-f758-4d40-b6a8-2d4cf7486f59 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.689167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 180/360] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:46.689778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c17e5-f758-4d40-b6a8-2d4cf7486f59 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.717209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:46.723629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540be8e-3d4d-4e22-9555-43b85c2db8a8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.775693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.775693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.775953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.776437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.792864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:46.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:46.799565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 11:02:46.800706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:46.801414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 181/361] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:46.801620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.836338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:46.841679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c17e5-f758-4d40-b6a8-2d4cf7486f59 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.890958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.891901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.892393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.892840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.904393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:46.909168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:46.910084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:46.910896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:46.913866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 181/362] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:46.913866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:46.942925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5b98c-10bb-4926-ba80-d7167daec361 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:46.993821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b6f58-19ff-4f06-bc97-d07b6055213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:46.994461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b6f58-19ff-4f06-bc97-d07b6055213e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:46.994747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b6f58-19ff-4f06-bc97-d07b6055213e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:46.995017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b6f58-19ff-4f06-bc97-d07b6055213e None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:46.997178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b6f58-19ff-4f06-bc97-d07b6055213e None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:46.999419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b6f58-19ff-4f06-bc97-d07b6055213e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:47.001471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 182/363] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:47.002915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.003403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.003862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.023031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:47.024561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:47.046683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d9b01-4eb3-41b5-88a6-0efe7dff4cdb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.046876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:47.095939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.095939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.095939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.095939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.096850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:47.096850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:47.098183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:47.104532 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 182/364] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:46 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:47.105612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.127453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 183/365] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:47.129612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.144693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:47.321879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d18805-d4bd-4026-8080-8bc8ebd9d072 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.326342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c51bb7-834b-496a-ac0e-bb9f680fbf16 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.384759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.385376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.385653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.385922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.386588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:47.386890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:47.390512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:47.394686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.395295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.395574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.395845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.396539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:47.396838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:47.405317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:47.406257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 183/366] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:47.406887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.422973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 184/367] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:47.423646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.458566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:47.564163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:47.593793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e4dd-9fb7-4bee-84c9-84f8338ef01e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.610068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c5b3e-5af4-4507-b28c-871483a40a3a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.647571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.648094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.648405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.648678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.649421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:47.649722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:47.651180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:47.658755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 184/368] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:47.659229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.661197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.662083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.662557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.663273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] PATH_INFO: `/v3/users/5a5b4cb008f24bba8f208ee46041a185` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.665818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5b4cb008f24bba8f208ee46041a185)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:47.667554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:47.672846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 185/369] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/users/5a5b4cb008f24bba8f208ee46041a185 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:47.676225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.794836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:47.797096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d5d7c-ee64-4b01-8a5c-5002911e0a54 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.815296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1268e0b-528c-41f3-89e5-77ebb8a61339 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:47.834568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.835194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.835476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.835751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.858363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:47.864455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:47.865318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:47.865695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:47.866059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:47.888750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:47.895066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:47.896127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:47.896946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:47.897610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 186/370] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:47.898070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:47.923094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:47.924934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.044424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 185/371] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:48.045703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.069667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45310780-460e-4b84-8a48-a134f0f91d80 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.101146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:48.152908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:48.153470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:48.153745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:48.154013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:48.155348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] RBAC: Adding query filter params (role.id=334460cecea042219ed608f1b5ff9096, user.id=5a5b4cb008f24bba8f208ee46041a185, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:48.155674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:48.157142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.160124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483adef-9b71-41fe-ab5d-a0424ac2a5aa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.179721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 187/372] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Aug 18 11:02:47 2022] GET /identity/v3/role_assignments?user.id=5a5b4cb008f24bba8f208ee46041a185&scope.system=all&role.id=334460cecea042219ed608f1b5ff9096 => generated 510 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:48.180768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.186992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:48.201915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:48.202922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:48.203424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:48.203919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:48.204822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:48.205345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:48.207041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.218051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 186/373] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:48.225057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.380248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc0f0f-2965-4934-8ec0-56125148339e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.395659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:48.421446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:48.422453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:48.422803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84611-1d20-4536-aa54-9c539056532d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.423173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:48.423644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:48.500959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:48.502792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.518171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:48.518828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:48.519480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:48.519480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:48.524128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:48.525607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.527620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 188/374] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:48.534721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.612851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 187/375] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:48.619366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.660593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:48.682101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:48.734746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177a49-f55d-42ce-b322-a00eac5399db None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.758533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d254f-f6bd-4f53-9941-3f8538a1a5ef None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.789761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:48.790495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:48.790776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:48.791044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:48.796270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:48.799471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.803056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 189/376] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:48.804512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:48.805678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:48.806175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:48.806645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] PATH_INFO: `/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:48.808231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.809410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] RBAC: Authorizing `identity:get_project(project_id=c8a7d63a241745a88c5287cb0d12f0c9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:48.812275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:48.817507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 188/377] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:48.827537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:48.947545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:48.949492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a495e-ed9a-40ad-8c6d-f80ee359aa5e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:48.986190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1198b-bf4d-43a2-ae6e-afa8a1bb1160 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:49.004273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:49.007120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.007120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.007120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.007120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:49.007120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:49.009143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506c182-543c-4ae5-a3d5-3a26b7034414 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:49.024214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:49.042042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa380e-7d2f-4947-9117-a539a3466a2c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.042333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa380e-7d2f-4947-9117-a539a3466a2c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.042604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa380e-7d2f-4947-9117-a539a3466a2c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.044350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 191/379] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:49.049028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:49.049699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.049990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.050155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183b113-5da7-4350-9723-8fe353198805 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.050252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] PATH_INFO: `/v3/groups/bced98ec26a24d67948f9b286abdd6b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.050449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183b113-5da7-4350-9723-8fe353198805 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.050733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183b113-5da7-4350-9723-8fe353198805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.062414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] RBAC: Authorizing `identity:get_group(group_id=bced98ec26a24d67948f9b286abdd6b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:49.063910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:49.066237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 189/380] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:48 2022] GET /identity/v3/groups/bced98ec26a24d67948f9b286abdd6b3 => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:49.068546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:49.072869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:49.085630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7183b113-5da7-4350-9723-8fe353198805 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:49.092043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7183b113-5da7-4350-9723-8fe353198805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:49.221811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 192/381] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:49.263583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66cd52-9e52-45d0-b504-d44c34195010 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:49.316131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:49.316965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.317233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.317504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.318225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] RBAC: Adding query filter params (group.id=bced98ec26a24d67948f9b286abdd6b3, role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=c8a7d63a241745a88c5287cb0d12f0c9) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:49.318520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:49.320137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:49.329932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 190/382] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:49 2022] GET /identity/v3/role_assignments?group.id=bced98ec26a24d67948f9b286abdd6b3&scope.project.id=c8a7d63a241745a88c5287cb0d12f0c9&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:49.531390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183b113-5da7-4350-9723-8fe353198805 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:49.653166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183b113-5da7-4350-9723-8fe353198805 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:49.689398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:49.703562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:49.704627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.705122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.705603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.727247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:49.734401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:49.735354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:49.736180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:49.738852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 193/383] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:49.744326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:49.756800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:49.864575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ec063-d06f-4eb6-af63-1db80acc1aba None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:49.910068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:49.910688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:49.910968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:49.911239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:49.911906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:49.912232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:49.913608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7394c4-815c-4a03-af29-3e9bcda42185 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:49.926474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 191/384] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:49.931830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:50.044955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b288b-ec5d-4251-825b-419b40189ce6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.045146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b288b-ec5d-4251-825b-419b40189ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.045430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b288b-ec5d-4251-825b-419b40189ce6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.047621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 192/385] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:50.057642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.057927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.058200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.071617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a2647-cbad-4152-8252-ae38de70728f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:50.086461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:50.087412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:50.118988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:50.119995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.120486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.120967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.122638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:50.124923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:50.127148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 194/386] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:49 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:50.136623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:50.177815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:50.178040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 193/387] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:50.227251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:50.280099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612437-e5a5-48b0-846a-62233c9af34f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:50.324078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:50.325057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.325567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.326027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.332936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:50.334669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:50.337046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 195/388] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:50 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:50.343278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:50.486942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38eb7a-4b02-45ca-9d2c-5e98776e3afb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:50.533836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c0b73-e4bf-40e4-b5b0-5eed6bf5791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:50.534500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c0b73-e4bf-40e4-b5b0-5eed6bf5791b None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.534776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c0b73-e4bf-40e4-b5b0-5eed6bf5791b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.535047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c0b73-e4bf-40e4-b5b0-5eed6bf5791b None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.537038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c0b73-e4bf-40e4-b5b0-5eed6bf5791b None admin] RBAC: Authorizing `identity:create_grant(project_id=8d671114d0af4f33841a2dc83ed32dc3, group_id=f3cbfea535fd4c6d90ffa377b316abff, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:50.538482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c0b73-e4bf-40e4-b5b0-5eed6bf5791b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:50.568038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 194/389] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Aug 18 11:02:50 2022] PUT /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:50.577444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:50.663730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd4f6e-3d70-4777-b71a-3f4fd1cd4fb6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.664080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd4f6e-3d70-4777-b71a-3f4fd1cd4fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.664310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd4f6e-3d70-4777-b71a-3f4fd1cd4fb6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.666457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 195/390] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:50.672315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.672627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.672865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.706490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed477e34-fcfa-4813-8a42-e9a903e4a23c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:50.712694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:50.716027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:50.769079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:50.788654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:50.789667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:50.791179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:50.791754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:50.817445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:50.820833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:50.825949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:50.827188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:50.828514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 196/391] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:50.835190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:50.836975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 196/392] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:50.965550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99e43a-fb01-4ebd-9cca-5132135603ce None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.007586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.008202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.008495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.008769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.009443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:51.009748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:51.011123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:51.026306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 197/393] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:51.033297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.147597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9b256-70ab-4176-93ed-77aaffe37c37 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.191783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.192859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.193373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.193883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.195236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:51.196930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:51.198716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 197/394] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:51 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:51.204454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.215259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.278607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:51.309337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:51.327218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33907b-574c-455a-b139-70afc61697c7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.374158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.374816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.375107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.375368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.398171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:51.402964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:51.403858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:51.405296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:51.405898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 198/395] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:51.412035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.417030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2290a4-f612-4f24-8746-891df3c04dbd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.465704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.466720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.468342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.468871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.470253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:51.471951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:51.474148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 198/396] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:51 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:51.479883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.569834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020e5a4-f17a-4d49-8541-a620cd2320e9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.591647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:51.613197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.613805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.614077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.614345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.615358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:51.615659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:51.617506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfc46-cb57-4ad0-ae69-6216c8f200b9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:51.626308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7721-f4c4-446e-89a6-f9ab3b7fc3dd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.632446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 199/397] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:51.632446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43a28b-947f-40a4-83e7-588e9a8b5cba None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.632446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43a28b-947f-40a4-83e7-588e9a8b5cba None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.632446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43a28b-947f-40a4-83e7-588e9a8b5cba None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.637341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 200/398] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:51.638277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43a28b-947f-40a4-83e7-588e9a8b5cba None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.656893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:51.680146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b9552-f808-40ca-8839-78e24bf2e1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.681170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9552-f808-40ca-8839-78e24bf2e1d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.681873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9552-f808-40ca-8839-78e24bf2e1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.683252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9552-f808-40ca-8839-78e24bf2e1d7 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.685723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b9552-f808-40ca-8839-78e24bf2e1d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d671114d0af4f33841a2dc83ed32dc3, group_id=f3cbfea535fd4c6d90ffa377b316abff, role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:51.687986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b9552-f808-40ca-8839-78e24bf2e1d7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:51.706247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 199/399] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Aug 18 11:02:51 2022] PUT /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:51.707786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fc85e-b56b-43d2-8daf-f7a3292a87e3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.708385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fc85e-b56b-43d2-8daf-f7a3292a87e3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.708883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fc85e-b56b-43d2-8daf-f7a3292a87e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.757422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-453fc85e-b56b-43d2-8daf-f7a3292a87e3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:51.758662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-453fc85e-b56b-43d2-8daf-f7a3292a87e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:51.790448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43a28b-947f-40a4-83e7-588e9a8b5cba None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.834834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:51.835802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.836429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.836984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.838720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:51.840468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:51.843664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 201/400] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:51 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:51.849994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:51.913963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 200/401] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:51.931861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:51.961763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74960bc4-df69-45c2-9302-63c7a75d6430 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.962094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74960bc4-df69-45c2-9302-63c7a75d6430 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.962414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74960bc4-df69-45c2-9302-63c7a75d6430 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.964080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 201/402] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:51.968841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:51.969400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:51.969400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:51.987934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc84444-b727-430f-b98f-ad5a08ecf778 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:51.988265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:51.989379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:52.046038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.047471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.047920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.048641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.050408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:52.052592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:52.055063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 202/403] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:51 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:52.061567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.071526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 202/404] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:02:52.167211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a6af-74ad-4777-b0dd-08681b3ca680 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:52.184522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:52.203322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.203985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.204350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.204628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.206589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] RBAC: Authorizing `identity:create_grant(project_id=28bb0208f1ed49099fb0dd0190df3d6b, group_id=f3cbfea535fd4c6d90ffa377b316abff, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:52.208038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:52.224655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 203/405] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Aug 18 11:02:52 2022] PUT /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:02:52.300264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.361934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:52.402201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.444736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68742c5e-c750-4f5c-852f-9508f5187866 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:52.487562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.489403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.489827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.490101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.505161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:52.511272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:52.512145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:52.512977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 203/406] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:52.519215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.554841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084c407-46c2-47b4-b494-48751e186375 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:52.624002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.624817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.625088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.625355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.648551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:52.656667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:52.658720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:52.660255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 204/407] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:52.667364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.704674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:52.712413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939a0c0-4d7f-4948-bd66-24b09158358b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:52.742243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:52.785692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.786295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.786567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.786832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.789039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:52.789349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:52.790726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:52.813075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 204/408] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:52.821769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.871218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f084304-986e-4a53-b109-2a88bf8b61e9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:52.925690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.926650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.927220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.927686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.928667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:52.929169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:52.930843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57278bd3-01bd-484b-98a5-e41a39578c4f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:52.945606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 205/409] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:52.947190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce2d6d-eea9-48e4-baa3-58c164670d29 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.947382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959a6e-1853-4db3-8017-802abb3791f3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:52.947865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce2d6d-eea9-48e4-baa3-58c164670d29 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.948369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce2d6d-eea9-48e4-baa3-58c164670d29 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.950258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 206/410] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:52.951146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce2d6d-eea9-48e4-baa3-58c164670d29 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:52.984364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143e8c7-2d52-4d66-93fd-cf530bea0b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:52.985001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143e8c7-2d52-4d66-93fd-cf530bea0b64 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.985246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143e8c7-2d52-4d66-93fd-cf530bea0b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.985515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143e8c7-2d52-4d66-93fd-cf530bea0b64 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:52.986704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143e8c7-2d52-4d66-93fd-cf530bea0b64 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:52.988235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143e8c7-2d52-4d66-93fd-cf530bea0b64 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:52.989852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 205/411] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:52 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:52.995561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:52.995829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:52.996104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.019555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:53.020780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:53.050933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:53.075468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:53.082389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce2d6d-eea9-48e4-baa3-58c164670d29 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.108161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 206/412] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:53.108161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:53.138618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.139646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.140162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.140805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] PATH_INFO: `/v3/groups/bced98ec26a24d67948f9b286abdd6b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.142232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] RBAC: Authorizing `identity:get_group(group_id=bced98ec26a24d67948f9b286abdd6b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:53.143907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:53.152734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 207/413] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:52 2022] GET /identity/v3/groups/bced98ec26a24d67948f9b286abdd6b3 => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:53.158782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:53.215888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e0d87-8822-47e3-ab9c-b2ed8b75a97b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.252785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.260433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.260433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.260433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.260433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:53.260433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:53.263186 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 207/414] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:53 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:53.264876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:53.293802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:53.345783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f42ee5-1784-4565-a6a7-59974a04d0e4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.393325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.393957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.394280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.394545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] PATH_INFO: `/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.395882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] RBAC: Authorizing `identity:get_project(project_id=c8a7d63a241745a88c5287cb0d12f0c9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:53.397408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:53.399021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 208/415] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:53 2022] GET /identity/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9 => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:53.410772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:53.444857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978a1cb-19e0-4a7f-a277-123d11cfce23 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.470546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:53.485703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.486688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.487192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.487660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.490118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] RBAC: Authorizing `identity:create_grant(project_id=28bb0208f1ed49099fb0dd0190df3d6b, group_id=f3cbfea535fd4c6d90ffa377b316abff, role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:53.491680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:53.514683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 208/416] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Aug 18 11:02:53 2022] PUT /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b/groups/f3cbfea535fd4c6d90ffa377b316abff/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:53.575565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:53.603240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692c1c-6c69-4f51-ab51-5cddfc82a36d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.640756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.641441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.641721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.641996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] PATH_INFO: `/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9/groups/bced98ec26a24d67948f9b286abdd6b3/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.644060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] RBAC: Authorizing `identity:create_grant(project_id=c8a7d63a241745a88c5287cb0d12f0c9, group_id=bced98ec26a24d67948f9b286abdd6b3, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:53.645574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:53.660316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 209/417] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Aug 18 11:02:53 2022] PUT /identity/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9/groups/bced98ec26a24d67948f9b286abdd6b3/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:02:53.743656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:53.765148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a906562-1005-47f1-9768-5e4e8f51a302 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.805577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.806225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.806505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.806779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.820421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:53.825267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:53.826160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:53.827259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 209/418] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:53.834447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:53.937254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:53.944388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50544-6654-41ce-a9d4-514758579bcc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:53.980788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:53.981402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:53.981684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:53.981959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:53.983211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:53.983530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:53.984938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc015369-0e60-4a1c-93c8-c597ca119830 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:53.994161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 210/419] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:54.000385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:54.049740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1603af-00ef-48e3-8873-3fc847b7ad12 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.050372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1603af-00ef-48e3-8873-3fc847b7ad12 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.050852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1603af-00ef-48e3-8873-3fc847b7ad12 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.052652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 211/420] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:54.057812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.058346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.058827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.077907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:54.079912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:54.196156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fb65-ea39-47c0-ae99-132aff66f87c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:54.201103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 212/421] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:54.260898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:54.261519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.261797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.262068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.284278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:54.351518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:54.353065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:54.476654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 210/422] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:54.483246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:54.542840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:54.610128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:54.611699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:54.629188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe634ab-78dd-4484-a8e6-cb142ba0aa15 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:54.665861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008cc4-7641-4673-b149-328531751b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:54.666587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008cc4-7641-4673-b149-328531751b17 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.666861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008cc4-7641-4673-b149-328531751b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.667133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008cc4-7641-4673-b149-328531751b17 None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.668398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008cc4-7641-4673-b149-328531751b17 None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:54.669830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008cc4-7641-4673-b149-328531751b17 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:54.672137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4d7c-cb56-417f-9b88-3c4f74c96532 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:54.676661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 213/423] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:54 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:54.677937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f578f04-7444-4ecd-b77e-ba73a2a89931 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.678219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f578f04-7444-4ecd-b77e-ba73a2a89931 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.678488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f578f04-7444-4ecd-b77e-ba73a2a89931 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.684135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 214/424] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:54.684929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f578f04-7444-4ecd-b77e-ba73a2a89931 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:54.710468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11522caa-a002-49cc-801f-dbbd0c85fff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:54.711453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522caa-a002-49cc-801f-dbbd0c85fff7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.711945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522caa-a002-49cc-801f-dbbd0c85fff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.712449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522caa-a002-49cc-801f-dbbd0c85fff7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.723533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11522caa-a002-49cc-801f-dbbd0c85fff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-11522caa-a002-49cc-801f-dbbd0c85fff7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:54.728710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:54.729593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:54.730387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:54.738202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 211/425] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:54.739711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.740219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.740721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.763162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:54.765729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:54.798271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f578f04-7444-4ecd-b77e-ba73a2a89931 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:54.848048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:54.848910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:54.849188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:54.849459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:54.850703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:54.852124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:54.854793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 215/426] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:54 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:54.858674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:54.861469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 212/427] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:54.862364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:54.892437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:54.902552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:54.991543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494364b-8c1e-4a18-8231-a2a4b155da13 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.034911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7829478-b879-435b-abcb-6a0970b7649c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.048952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.050066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.050850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.051599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.052562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:55.053073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.055190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.065747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 216/428] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:55.082282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.095163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.095818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.096100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.096414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.097126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:55.097430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.098961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.105500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 213/429] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:54 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:55.189605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:55.208540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb790a-4cf2-49e5-867f-e6378fa33bec None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.249626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.250505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.251002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.251434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.263174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.336151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.338246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.384921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f7c0-62af-4ad4-8273-eb4ac6ae01e7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.395379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:55.430284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.431497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.431982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.432482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.445117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:55.449047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:55.453256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:55.454138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 11:02:55.454928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:02:55.458413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 214/430] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:55.472291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 217/431] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:02:55.473367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.477433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.608973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de922c70-be5a-447e-aefe-321cc98e5f7f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.624313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee729e-bbdc-4f9d-b361-30a7ba9c45bc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.646269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.646895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.647167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.647434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] PATH_INFO: `/v3/projects/8d671114d0af4f33841a2dc83ed32dc3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.648583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] RBAC: Authorizing `identity:get_project(project_id=8d671114d0af4f33841a2dc83ed32dc3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.650020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.652060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 218/432] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/projects/8d671114d0af4f33841a2dc83ed32dc3 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:55.658345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.668753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.669629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.670201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.670778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.673661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:55.674356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.675975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.689196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 215/433] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:55.695116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.728615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:55.735634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:55.806867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cef3e-8d16-4c3d-85cc-d7527ec4c75d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.831951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53065cdd-8403-4c50-9f8b-40dc3960ddad None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:55.849112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.850004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.850481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.850897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.852221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.853791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.855876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 219/434] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:55.862632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:55.876760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:55.877915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:55.878273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:55.878616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:55.976261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:55.976449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:55.992881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ea931-2f66-4746-8d88-cbf2b2f672e5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.010404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.016244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.036409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.037022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.037303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.037904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.038642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=8d671114d0af4f33841a2dc83ed32dc3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:56.038953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:56.040424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac58b2-7995-47cb-be19-79d624b8bc8b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:56.046941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 220/435] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=8d671114d0af4f33841a2dc83ed32dc3&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:56.048101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d7eb3-1700-4687-9c8e-0b82b071390c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.048415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d7eb3-1700-4687-9c8e-0b82b071390c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.048693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d7eb3-1700-4687-9c8e-0b82b071390c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.052225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 221/436] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:02:56.055442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8dc5-89d5-4a27-a30d-d4e650b93fac None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.055725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8dc5-89d5-4a27-a30d-d4e650b93fac None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.055999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8dc5-89d5-4a27-a30d-d4e650b93fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.081858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-babd8dc5-89d5-4a27-a30d-d4e650b93fac None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:56.083195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-babd8dc5-89d5-4a27-a30d-d4e650b93fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:56.115423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 216/437] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:02:56.123445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.176281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 222/438] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:56.187550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039886-0a45-477f-8d61-8bb5156df300 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.187926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039886-0a45-477f-8d61-8bb5156df300 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.188316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32039886-0a45-477f-8d61-8bb5156df300 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.194114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 223/439] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:02:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:02:56.195537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.195897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.196288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.215775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:02:56.217646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:02:56.242865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d8acc-1cbc-47a2-98ab-5d501405d6c5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.270743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.287855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.288524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.288801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.289072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.290195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:56.291649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:56.293338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 217/440] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:56.300050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.306791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 224/441] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:02:56.417041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431335-5c76-4813-ac27-30e1f8c1b525 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.454875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.455824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.456490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.456771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.457045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.458168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:56.459633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:56.461262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 218/442] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:56.469311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.514889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.546942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.587363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852480d2-6bf2-438f-813a-268cca0ed20d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.595715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cee275-6ddf-4c12-bd33-93c92b4d2d3a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.622274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.622898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.623215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.623580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.631302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.631902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.632181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.632483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.633204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=866bc5028f7b4637aa1ae91761cd03b5, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:56.633469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:56.633567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:56.634938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:56.638215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:56.639213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:56.639998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 225/443] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:02:56.641152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.643581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 219/444] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=866bc5028f7b4637aa1ae91761cd03b5 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:56.646006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.755340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33c3c-0bad-468a-b61d-7d5db86c0781 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.757829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06405511-b996-435a-ae80-03eb340c9240 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.757982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.785121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:56.797924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.798549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.798844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.799110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.803395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.803992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.804318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.804593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.805258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:56.805563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:56.807023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:56.809783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:02:56.814766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:02:56.815674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:02:56.816938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 226/445] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:02:56.818703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 220/446] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:56.821563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.825794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:56.935920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6ea06-25a7-4532-9bb6-5d2290a856f7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.938613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad5e50-460f-457d-bc58-0b459114504c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:56.976096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.976462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.976856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.977131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.980412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:56.981019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:56.981300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:56.981576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:56.982239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:56.982606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:56.984352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:56.996294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 227/447] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:57.001557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.035977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.044916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.061494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.063218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.111991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0f53-401d-4d9c-8c70-bf18249c93a9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.146086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.146684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.147003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.147232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.162412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 221/448] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:02:57.168321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.212600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.214033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.239239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.250129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.266722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789147a5-1bfb-41b8-bf5e-800e89d9389b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.301731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.302285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.302623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.302893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] PATH_INFO: `/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.304024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] RBAC: Authorizing `identity:get_project(project_id=28bb0208f1ed49099fb0dd0190df3d6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.305483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.307092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 222/449] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/projects/28bb0208f1ed49099fb0dd0190df3d6b => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:57.312524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.317714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 228/450] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:02:57.318675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.411116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3782856-02c3-406a-a83d-257bea913a33 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.421894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fb45f-0154-426f-9a30-ca6c26bae08e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.445870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.446487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.446764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.447041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] PATH_INFO: `/v3/groups/f3cbfea535fd4c6d90ffa377b316abff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.447166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.448575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] RBAC: Authorizing `identity:get_group(group_id=f3cbfea535fd4c6d90ffa377b316abff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.450047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.452388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 223/451] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/groups/f3cbfea535fd4c6d90ffa377b316abff => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:57.456663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.457279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.457554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.457821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] PATH_INFO: `/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.458037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.458933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] RBAC: Authorizing `identity:get_project(project_id=c8a7d63a241745a88c5287cb0d12f0c9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.460381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.461974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 229/452] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/projects/c8a7d63a241745a88c5287cb0d12f0c9 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:02:57.467140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.479745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.558723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c10a7-6949-4541-a290-f6b34a313a34 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.566912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939286-e95d-4914-8663-9e7bcda3bcc1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.591735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.592945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.593219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.593488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.594197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] RBAC: Adding query filter params (group.id=f3cbfea535fd4c6d90ffa377b316abff, role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:57.594501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.595930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.600812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11892a1d-1a25-40d6-9bf4-77e3494866a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.601429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892a1d-1a25-40d6-9bf4-77e3494866a1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.601518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 224/453] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/role_assignments?group.id=f3cbfea535fd4c6d90ffa377b316abff&scope.project.id=28bb0208f1ed49099fb0dd0190df3d6b&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:57.601780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892a1d-1a25-40d6-9bf4-77e3494866a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.602054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11892a1d-1a25-40d6-9bf4-77e3494866a1 None admin] PATH_INFO: `/v3/groups/bced98ec26a24d67948f9b286abdd6b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.603216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11892a1d-1a25-40d6-9bf4-77e3494866a1 None admin] RBAC: Authorizing `identity:get_group(group_id=bced98ec26a24d67948f9b286abdd6b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.604684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11892a1d-1a25-40d6-9bf4-77e3494866a1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.609651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 230/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/groups/bced98ec26a24d67948f9b286abdd6b3 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:02:57.612389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:02:57.703350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:02:57.719947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad1dea-8f93-4cb8-94c8-8f4f7434650b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:02:57.755488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:02:57.756082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:02:57.756382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:02:57.756653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:02:57.757367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] RBAC: Adding query filter params (group.id=bced98ec26a24d67948f9b286abdd6b3, role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=c8a7d63a241745a88c5287cb0d12f0c9) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:02:57.757671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:02:57.759097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a75d6-7ce5-4c3b-8698-e672f5d25ac6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:02:57.765147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 225/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Aug 18 11:02:57 2022] GET /identity/v3/role_assignments?group.id=bced98ec26a24d67948f9b286abdd6b3&scope.project.id=c8a7d63a241745a88c5287cb0d12f0c9&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:00.017996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437d03-fd17-48e5-9175-85d82856c287 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.018528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437d03-fd17-48e5-9175-85d82856c287 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.018528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437d03-fd17-48e5-9175-85d82856c287 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.020121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 231/456] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:00.026521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd771f1-8ae2-4468-a52d-2a48534a3a69 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.026977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd771f1-8ae2-4468-a52d-2a48534a3a69 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.027163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd771f1-8ae2-4468-a52d-2a48534a3a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.053588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4fd771f1-8ae2-4468-a52d-2a48534a3a69 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:00.054796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4fd771f1-8ae2-4468-a52d-2a48534a3a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:00.156495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 226/457] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:00.282195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1acdb7-ac01-4dad-a52c-1036db96846a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.282793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1acdb7-ac01-4dad-a52c-1036db96846a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.283280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1acdb7-ac01-4dad-a52c-1036db96846a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.284836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 232/458] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:00.290603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab7d0e-6203-4167-8d88-a9ef55a8e6d6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.291119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab7d0e-6203-4167-8d88-a9ef55a8e6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.291595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab7d0e-6203-4167-8d88-a9ef55a8e6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.313565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68ab7d0e-6203-4167-8d88-a9ef55a8e6d6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:00.315083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68ab7d0e-6203-4167-8d88-a9ef55a8e6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:00.400582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 227/459] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:00.406614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78f6d-3525-4c2a-be47-a228197b7a9e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.407159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78f6d-3525-4c2a-be47-a228197b7a9e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.407652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78f6d-3525-4c2a-be47-a228197b7a9e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.410188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 233/460] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:00.414293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.414573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.414842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.436153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:00.437522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:00.439469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:00.497211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a78f6d-3525-4c2a-be47-a228197b7a9e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:00.526698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 228/461] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:00.596109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:00.650550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a78f6d-3525-4c2a-be47-a228197b7a9e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:00.697076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:00.697741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.698024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.698295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.712876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:00.714328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:00.740720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:00.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:00.742596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:00.742596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:00.742596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:00.742596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 234/462] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:00.747932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:00.851627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:00.866825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40524861-177e-477d-9d8f-ec9b84ed432d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:00.902583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:00.903199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:00.903476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:00.903802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:00.905015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:00.905319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:00.907467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:00.918282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:00.926800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 229/463] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:00.931652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:00.990524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f067ab-0031-42ae-8c9e-64b2954b1a0c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.009702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:01.033881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80219c3e-abcb-435f-8f9a-996954a25c3d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.035875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d743df-a182-4669-97d9-2879928e830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.037138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d743df-a182-4669-97d9-2879928e830f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.037639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d743df-a182-4669-97d9-2879928e830f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.038164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d743df-a182-4669-97d9-2879928e830f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.050976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d743df-a182-4669-97d9-2879928e830f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.052987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d743df-a182-4669-97d9-2879928e830f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-82d743df-a182-4669-97d9-2879928e830f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:01.067991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:01.069160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:01.069984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:01.069984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:01.069984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:01.069984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 235/464] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:01.069984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d743df-a182-4669-97d9-2879928e830f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:01.071184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.071813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.072091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.073327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.084915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.086378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-903286d5-330c-4083-a847-d45fc3ec73e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:01.102378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:01.103272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:01.104125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:01.104125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:01.104125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:01.104125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 230/465] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:01.104891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fcb1e-e748-4aca-a3d3-0ce474b5553c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.104958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fcb1e-e748-4aca-a3d3-0ce474b5553c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.105221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fcb1e-e748-4aca-a3d3-0ce474b5553c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.106600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 231/466] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:01.107133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fcb1e-e748-4aca-a3d3-0ce474b5553c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:01.166014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:01.194056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d743df-a182-4669-97d9-2879928e830f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.220416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fcb1e-e748-4aca-a3d3-0ce474b5553c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.232058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.232846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.233130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.233645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.234403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.235838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.255636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.256269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.256554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.256827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.257468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:01.257771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.268513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 236/467] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:03:01 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:01.268716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819fab-22ac-494a-aacb-5f9a2ab9d3b6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.268832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:01.283043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 232/468] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:01.289310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.289670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.290153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.298699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:01.313487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:01.314870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:01.396427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccef81c-6123-4b1c-8be2-3c4a9657fa1b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.408639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 233/469] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:01.409833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:01.437713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.438355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.438640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.438913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.439564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:01.439867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.442184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.455798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 237/470] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:01.462522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:01.474690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:01.557009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86dce6-e010-4639-b610-ef7195aa5cd1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.559930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:01.584830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3bb8b-26a6-4c47-a52b-d755eff35583 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.610273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.611146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.611428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.611690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.612606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.614799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.625947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acc588-e640-40ba-9355-bf4d04fbbff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.626565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acc588-e640-40ba-9355-bf4d04fbbff3 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.626843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acc588-e640-40ba-9355-bf4d04fbbff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.627117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acc588-e640-40ba-9355-bf4d04fbbff3 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.627772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acc588-e640-40ba-9355-bf4d04fbbff3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.629256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acc588-e640-40ba-9355-bf4d04fbbff3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.644544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 234/471] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:03:01 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:01.660364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 238/472] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:03:01 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:01.762603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:01.826369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099c521-e114-484e-bc19-0255a4a25ef2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.826744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099c521-e114-484e-bc19-0255a4a25ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.827017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099c521-e114-484e-bc19-0255a4a25ef2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.829417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 239/473] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:01.834036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.834320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.834588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.860075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:01.861426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:01.876618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:01.887158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984236f5-6127-44c7-9141-8ba2ee079e88 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:01.931844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:01.932470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:01.932750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:01.933023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:01.947221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:01.948698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 240/474] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:01.948807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:01.965676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:01.966636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:01.967976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:01.967976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:01.967976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:01.967976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 235/475] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:01.973092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:02.044995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:02.080318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87ea4d-2781-43b9-9a64-231ea8e3d2eb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:02.116140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:02.116834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:02.117120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:02.117392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:02.118110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:02.118421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:02.119802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:02.132273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 241/476] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:02.138073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:02.275021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ddf63-3911-417c-8b2e-78c3b2c600a9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:02.311240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:02.311916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:02.312223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:02.312554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:02.313214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:02.314659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:02.337121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 236/477] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:03:02 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:03:02.370967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:02.486998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde21576-6052-4495-a681-d85defba72c4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:02.517166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:02.526018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:02.526559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:02.526838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:02.527111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:02.527787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:02.529190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:02.539696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.sql.core [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 11:03:02.539696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 11:03:02.539696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [parameters: {'id': '7bb280a8a8f4443b8aad3363bb8cf158', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 11:03:02.539696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Aug 18 11:03:02.556916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return ref.to_dict() Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 11:03:02.557805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application yield resource Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise self.value Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application session.commit() Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 11:03:02.558736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.flush() Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application compat.raise_( Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 11:03:02.559601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 11:03:02.561081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 11:03:02.562166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [parameters: {'id': '7bb280a8a8f4443b8aad3363bb8cf158', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:02.562963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 11:03:02.565126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:02.567407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 242/478] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Aug 18 11:03:02 2022] POST /identity/v3/roles => generated 144 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:02.567544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:02.581481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:02.684654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af491a7-95c8-44b6-8a92-8602c44a3e7f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:02.723981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb2563-4852-424b-b53f-20a1cdd75e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:02.724642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb2563-4852-424b-b53f-20a1cdd75e5d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:02.724922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb2563-4852-424b-b53f-20a1cdd75e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:02.725196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb2563-4852-424b-b53f-20a1cdd75e5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:02.736793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb2563-4852-424b-b53f-20a1cdd75e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1acb2563-4852-424b-b53f-20a1cdd75e5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:02.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:02.746139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:02.747339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 237/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:02.752717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:02.866114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1395b-e0e4-4b06-9039-016b4d10b3ca None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:02.903825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:02.904873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:02.905381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:02.905852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:02.906721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:02.907235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:02.908848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd6293-729e-4987-9dc9-fd4b77f4a1a3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:02.918164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 243/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:03.950781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320458-4383-4652-bd3c-bc09bb572765 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:03.951914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320458-4383-4652-bd3c-bc09bb572765 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:03.952568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320458-4383-4652-bd3c-bc09bb572765 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:03.954486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 238/481] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:03.959725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b7926-922d-406d-9fc5-6557b9278a49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:03.960663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b7926-922d-406d-9fc5-6557b9278a49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:03.961141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b7926-922d-406d-9fc5-6557b9278a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:03.984632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ab2b7926-922d-406d-9fc5-6557b9278a49 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:03.985837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ab2b7926-922d-406d-9fc5-6557b9278a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:04.064560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:04.077340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 244/482] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:04.155402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c153915-377d-4919-8be1-493963269381 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.155740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c153915-377d-4919-8be1-493963269381 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.155946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c153915-377d-4919-8be1-493963269381 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.157588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 239/483] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:04.162592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.162878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.163151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.189551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:04.190756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:04.235520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6669272-96f7-4d36-94ba-5398cf6b594f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.235794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6669272-96f7-4d36-94ba-5398cf6b594f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.235979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6669272-96f7-4d36-94ba-5398cf6b594f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.237588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 240/484] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:04.245120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.245405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.245676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.271518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 245/485] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:04.285233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:04.286453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:04.374103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:04.383334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 241/486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:03:04.471431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:04.575029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23840e31-58a7-47d3-90d4-3914b18c1746 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:04.593964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:04.596477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:04.615335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:04.615967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.616269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.616541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.628314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:04.632989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:04.633882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:04.634671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 246/487] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:04.640273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:04.759359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1993d4-46d1-4efa-91e6-3d12e99e3cb7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:04.801711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59f94-3d8e-4fdc-a678-0d7cf4764228 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:04.812085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:04.813275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.813559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.813980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.832828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:04.838825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:04.839770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:04.840604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:04.841196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:04.841282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 242/488] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:04.841347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.841418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:04.841485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.841485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:04.842321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:04.848233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:04.848233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:04.856547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 247/489] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:04.857599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:04.897305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:04.905116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:04.952470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f3c66-c0d3-427a-b24c-167d6f88877a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:04.973595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a503bc8-de45-472d-b163-7cfab2051c3c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:04.987502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:04.988131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:04.988430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:04.988697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.000935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:05.005741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:05.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:05.007407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 243/490] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:05.008050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.018767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.019380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.019659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.019926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.020631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:05.021388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.022772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377695d-363a-40cf-a700-1f5ece54fda8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.034229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 248/491] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:05.035307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c59d-e835-4620-962a-6d3bf3cff228 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.035599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c59d-e835-4620-962a-6d3bf3cff228 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.035870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c59d-e835-4620-962a-6d3bf3cff228 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.037311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 249/492] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:05.041058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7c59d-e835-4620-962a-6d3bf3cff228 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.128051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf418f-de91-4c17-93cf-5baf56df472b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.145904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.148805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.155630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7c59d-e835-4620-962a-6d3bf3cff228 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.169027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.169666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.170855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.171128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.182568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.184050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:05.200733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:05.201748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:05.202575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:05.202575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:05.202575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:05.202575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 244/493] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:05.202796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.204419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.205027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.205325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.205576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.206230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:05.206919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.208312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbda32-4655-4ea9-aa87-d2efbd75b994 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.221580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 250/494] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:05.221580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.221580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.221580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.239731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:05.240951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:05.317954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 251/495] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:05.318833 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.322579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf333a-9f19-4c82-b598-435cdd36c3ea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.359146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.359975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.360360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.360728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.372448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.374285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:05.389730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:05.390635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:05.391478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:05.391478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:05.391478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:05.391478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 245/496] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:05.392357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.413360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.432953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.444110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dadf6-923d-4e67-abc1-a22ead2bae1f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.481887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.483318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.483745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.484324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.485145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:05.485732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.487350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.500543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 252/497] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:05.501612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.509496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdab61-5c35-49bf-9b33-3c6bcf9a9dae None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.550193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.550828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.551098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.551372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.563001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.564509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f4287071-ab10-4fc7-925b-4be50159c8f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:05.580227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:05.581139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:05.581986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:05.581986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:05.581986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:05.581986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 246/498] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:05.581986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156599-75f0-42c5-b999-006007b82087 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.582259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156599-75f0-42c5-b999-006007b82087 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.582752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156599-75f0-42c5-b999-006007b82087 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.587803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 247/499] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:05.587803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156599-75f0-42c5-b999-006007b82087 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.616840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88163d26-d1c1-4bc4-8e16-57d475b7a060 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.649989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.650515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.650794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.651064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.651701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:05.652001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.653420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.660390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.665712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 253/500] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:05.666647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.678959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.694931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156599-75f0-42c5-b999-006007b82087 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.728173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.728818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.729102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.729376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.737384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.738174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-aeff3759-259a-44cd-a1d5-56f0566f23d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:05.750793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:05.751735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:05.752689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 248/501] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:05.753253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.753317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.753858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.768976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd356fa-e8a4-46bd-9afa-897edefd4e15 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.771557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:05.772776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:05.803620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.804240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.804554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.804827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.805465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:05.805903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:05.807269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:05.818639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 254/502] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:05.819248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.845672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 249/503] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:05.846848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:05.898556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.911182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:05.933911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563c1b-edc4-4e8d-b7d2-6e1052fe31fe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.958168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18db93-19cb-44f2-8c39-dabe82e8ff6d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:05.971131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.971756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.972039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.972445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.983835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:05.989598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:05.990454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:05.991313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:05.993677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:05.994304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:05.994583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:05.994854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:05.995538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 255/504] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:05.996311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.002768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.004235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:06.017020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:06.017922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 250/505] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:06.018844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.115471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9db53e-08eb-478c-9249-2af06b1b9ca7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.129572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08801e4-dff0-427a-96a9-5a19084983b7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.144232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.149329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.150426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.151228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.151514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.151785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.152540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:06.152842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.155934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.163711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.164304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.164580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.164844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.172424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.173869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:06.184626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:06.186043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 11:03:06.186922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:06.188333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 251/506] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:06.189340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.198038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 256/507] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:03:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:06.198038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.288764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a11845-e8f0-44ed-b2ff-e1b702a3ba41 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.294996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccad16-c91b-455f-8eb4-ee627b76be89 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.322577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.323177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.323455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.323721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.324413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:06.324722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.326157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.332115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.332889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.333166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.333437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.334362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:06.334465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 252/508] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:06.334585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.334973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.336122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.355697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 257/509] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:06.356559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.382540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.403010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.440717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97acb2-1462-4c93-a992-f3759e640d11 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.456730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf8a65-54fb-4cf8-af48-af28510dc46c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.473889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.474513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.474791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.475062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.485605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.487047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.490489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.491080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.491357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.491628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.492379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:06.492685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.494130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:06.501269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:06.502259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:06.503124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:06.503124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:06.503124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:06.503124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 253/510] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:06.503335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.515621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 258/511] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:06.516490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.600443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.607113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b9150-cb41-4c16-975e-f84aba19d2bb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.615182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a3057-4fe4-4be4-9044-5ffe944a70b4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.617477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.642046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.642643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.642918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.643193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.649771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.650430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.650711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.650987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.661490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.662955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:06.677661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:06.683829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:06.685222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:06.685222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:06.685222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:06.685222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 259/512] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:06.685222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.711450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.712918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.778742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042cbfb-a84f-4bfe-a214-1bcf783a300a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.802348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.809777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 254/513] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:06.810611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.816828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.817434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.817715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.817990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.843279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:06.892319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.893811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.911819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602d0a7-6627-40a1-a2bc-63c16f3163e6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:06.945559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:06.946148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:06.946424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:06.946693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:06.947325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:06.947623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:06.949041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:06.960845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 255/514] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:06.961685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:06.993819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 260/515] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:06.994666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.017855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.055355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.064792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352bf1b-e403-4339-a974-b646c27b685b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.097525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.098120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.098398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.098665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.098753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfbbc3-05d7-4772-8b64-619d22990a39 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.133197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.133786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.134056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.134322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.134952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:07.135246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.136691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.147680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 261/516] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:07.148561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.169331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.171742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.226990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.258646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac87e4-5367-435e-b1ea-e930a815ac49 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.271589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 256/517] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:07.272431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.278493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.293165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.293784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.294063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.294331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.305408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.306880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:07.321548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:07.322484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:07.323320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:07.323320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:07.323320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:07.323320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 262/518] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:07.323528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.374566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a264046-a143-4712-9ce5-1e83ac9895cb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.408199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.408817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.409090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.409356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.409974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.411395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.424711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c7f8-2d96-494c-a067-7ecd2677c356 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.436229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 257/519] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:03:07 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:07.437024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.450635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.464170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.464842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.465123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.465398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.478242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.479645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:07.496170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:07.497484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:07.498392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:07.498392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:07.498392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:07.503086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 263/520] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:07.504063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.528456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.564419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c683b25-28c2-4108-9efe-2c1e6cdbe33e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.600953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.601583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.601865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.602137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.610538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.611992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.621836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae92fc3-58f3-4e59-b820-86cdaa802074 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:07.624818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:07.625729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:07.626607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 258/521] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:07.627154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.660788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.661579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.661703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.661950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.674510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.675951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:07.691078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:07.692035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:07.692917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:07.692917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:07.692917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:07.692917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 264/522] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:07.693138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.714464 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.755211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33343de4-7932-48cb-9e25-2f239a99739b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.767305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.791016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.791618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.791904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.792175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.792854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:07.793193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.794558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.800658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33895723-55d3-46cf-a11e-48f1bbe18fb2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.811629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 259/523] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:07.811629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.840622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.841231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.841510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.842355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.843087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:07.843389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.844951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.858165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 265/524] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:07.859150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:07.926431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc7211-4644-4b5a-9732-7d2c97648225 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.955677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:07.962644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:07.963258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:07.963469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c3ca-bd04-4712-81e2-62f705fae61a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:07.963588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:07.963776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:07.964525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:07.964832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:07.966272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:07.995261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.002626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.003236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.003511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.003780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.004452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:08.004754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.006175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.007694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 260/525] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:08.012979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.023539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 266/526] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:08.023539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.127414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241660c-cc0d-49a2-9124-23d280ad5e11 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.129865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e2dec-e2e1-447e-b6e5-a66709ccd67e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.163071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.163696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.163976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.170354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.171411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.171624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.172034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.172589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.173125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.180337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.181956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:08.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:08.186371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:08.187268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 267/527] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:08.187735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:08.199387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:08.200339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:08.201328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:08.201994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 261/528] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:08.202564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.250782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.256429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.303488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261479fa-836f-414b-9046-bdf6dc0d0b92 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.318476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230776-dfa3-4613-a363-82ea1d3306a2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.340000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.340631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.340905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.341601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.359570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.360214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.360749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.361029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.370465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.372856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:08.387754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:08.388623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:08.389484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 262/529] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:08.389949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.417286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.419001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.481067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.502583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7138-d878-4a9d-b950-ca79385f42fc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.512329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.540156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 268/530] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:08.540156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.543138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.543772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.544052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.544354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.545747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:08.546123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.547575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.568918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 263/531] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:08.568918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.656874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6ae7b-97de-4965-8f3a-ec2e4deebf3d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.680335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e5c5e-56f3-4def-96a3-42c38dd3b5ed None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.694086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.694698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.694961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.695224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.695903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:08.696212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.697905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.714560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.715209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.715495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.715782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.716508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:08.717361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.718308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 269/532] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:08.719011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.723850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.733875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.742601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 264/533] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:08.742601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.769574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:08.835001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e55638-512d-47fa-9032-1dbd0d2c35d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.847136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399f1d0-7d51-4404-8f84-b473adc06621 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:08.872450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.873277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.873593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.873866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.882274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:08.882879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:08.883160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:08.883429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:08.884153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=aefa27e6907c4ba28483d31a68753c31) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:08.884489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.885287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:08.885925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.887029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:08.892171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 265/534] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/role_assignments?user.id=aefa27e6907c4ba28483d31a68753c31&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:08.892985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:08.913311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:08.914442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:08.915465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:08.915465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:08.915465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:08.923410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 270/535] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:08.924977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:09.013063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:09.027132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:09.043275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf6a86-2d5c-41b5-9ed6-3486dd067b11 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:09.073746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46eb17-5e67-40db-bb21-956df031e2fc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:09.082705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:09.083445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.083789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.084130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.084986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=1f43ff51c5f84d34978c46bce6db67ee) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:09.085361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:09.086872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:09.100301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 266/536] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/role_assignments?user.id=1f43ff51c5f84d34978c46bce6db67ee&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:09.101436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:09.132996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:09.134538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.136049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.136527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.139728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=5e78415ea98940bbbc59c733c8ffefa4) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:09.142363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:09.145406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:09.162998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 271/537] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:08 2022] GET /identity/v3/role_assignments?user.id=5e78415ea98940bbbc59c733c8ffefa4&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:09.258303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a97f99-84ed-4f18-b7cb-424a79c25d89 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:09.299849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:09.300819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.301305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.301767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.302662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:09.303145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:09.304826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055bea9-503c-4da6-a030-b74f53981e8e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:09.319632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 267/538] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:09.325704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:09.411804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627588-71e6-49ab-a5b8-9cd2aac8d3c4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.412414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627588-71e6-49ab-a5b8-9cd2aac8d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.412898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627588-71e6-49ab-a5b8-9cd2aac8d3c4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.417428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 268/539] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:09.423756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.424035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.424336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.445445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:09.446305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:09.472551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:09.479926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a1e12-a281-478d-a2f3-eb43423096a8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:09.535240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:09.552480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:09.553031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.553304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.553570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.564830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:09.566295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:09.582786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 269/540] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:09.593125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:09.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:09.594851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 272/541] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:09.599986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:09.763891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9a13-30de-4cb2-b499-ef6483c56a2d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:09.814123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:09.814693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:09.815093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:09.815428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:09.816274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:09.816585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:09.818209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:09.836494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 270/542] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:09.842109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:09.890761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:09.900047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:10.001140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b919a-6b76-42c1-9b83-c63d681c6573 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:10.018060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3679b8-ca2e-449a-a8aa-503b4a3cc633 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:10.070050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:10.070586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:10.070855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:10.071122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:10.087138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:10.087739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:10.088013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:10.088407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:10.089137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=a09c08eca5be4a1b8977ff91bb468358) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:10.089478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:10.090902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:10.094136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:10.100324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:10.101203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:10.101995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:10.104131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 273/544] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:09 2022] GET /identity/v3/role_assignments?user.id=a09c08eca5be4a1b8977ff91bb468358&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:10.104267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 271/544] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:10.110692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:10.190478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:10.267213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75583a1-5e21-46f3-90dd-669fd820df23 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:10.307973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:10.308621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:10.308901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:10.309173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:10.309840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:10.310142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:10.311562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:10.320440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 274/545] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:10.329252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:10.424584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:10.454658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159bfb-efc3-4c30-a37d-e6d3ebabc912 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:10.490251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:10.490873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:10.491143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:10.491406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:10.503694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:10.505165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:10.519789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:10.520715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:10.521568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:10.521568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:10.521568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:10.521568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 272/546] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:10.526988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:10.672313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a81381-f73d-4f0b-a862-8b3283c66b5e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:10.717102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:10.720817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:10.721456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:10.722196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:10.722683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:10.723515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:10.724108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:10.725514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:10.742921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 275/547] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:10.748418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:10.907530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb195c-3a4d-4ed0-bb0f-d06260980216 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:10.932673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:10.959416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:10.960416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:10.960925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:10.961399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:10.975677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:10.977406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0ae6de1a-d0f9-4afc-831c-f9ce77f34aea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:10.995494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:10.996458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:10.998344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:11.000859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 273/548] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:11.009292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:11.104372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1e0e9-11ae-4a0e-a729-0ed093e327f8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.105344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1e0e9-11ae-4a0e-a729-0ed093e327f8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.105873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1e0e9-11ae-4a0e-a729-0ed093e327f8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.107773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 274/549] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:11.112633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.113181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.113668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.140126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e46dae-0396-4859-b50d-fda97b350098 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:11.157249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:11.158797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:11.177476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:11.178086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.178363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.178631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.179366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:11.179670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:11.181126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:11.199844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 276/550] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Aug 18 11:03:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:11.205696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:11.239067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:11.286894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 275/551] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:11.311063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038edcd-978e-4e2b-9e6c-585b4de26c29 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:11.347378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ff1af-67e9-4d77-bc66-0b9dda71b992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:11.347984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ff1af-67e9-4d77-bc66-0b9dda71b992 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.348295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ff1af-67e9-4d77-bc66-0b9dda71b992 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.348568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ff1af-67e9-4d77-bc66-0b9dda71b992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.418338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ff1af-67e9-4d77-bc66-0b9dda71b992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:11.420400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ff1af-67e9-4d77-bc66-0b9dda71b992 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:11.452279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:11.528497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 277/552] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 11:03:11.533286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:11.673772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cbb9c-c7d2-48d0-a4b2-d78649339509 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.674121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cbb9c-c7d2-48d0-a4b2-d78649339509 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.674344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cbb9c-c7d2-48d0-a4b2-d78649339509 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.675931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 278/553] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:11.680556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.680842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.681108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.688314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcc823-b093-464f-8611-321a262ef0d1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:11.704778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:11.715149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:11.723370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:11.725992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:11.726784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.727206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.727618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.739441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:11.741154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:11.758312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:11.759561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:11.760620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:11.760620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:11.760620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:11.762636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 276/554] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:11.770055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:11.826420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 279/555] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:11.827710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:11.894372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e2a6-3f4e-48b7-adac-8f85089c3025 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:11.900583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:11.934521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ce93-8650-428b-9975-be5841974802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:11.935479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ce93-8650-428b-9975-be5841974802 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.935942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ce93-8650-428b-9975-be5841974802 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.936424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ce93-8650-428b-9975-be5841974802 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.937352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ce93-8650-428b-9975-be5841974802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:11.937991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ce93-8650-428b-9975-be5841974802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:11.939547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ce93-8650-428b-9975-be5841974802 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:11.947224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f655b7-c9a1-4e3e-be10-cc48b0612797 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:11.955967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 277/556] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:11.958350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598be71d-bdc6-45e1-b1ef-7499b9c5c954 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.959163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598be71d-bdc6-45e1-b1ef-7499b9c5c954 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.959717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598be71d-bdc6-45e1-b1ef-7499b9c5c954 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.967323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 278/557] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:11.969571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598be71d-bdc6-45e1-b1ef-7499b9c5c954 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:11.991289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3ad4-8571-41e0-9995-85b08d340b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:11.991926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3ad4-8571-41e0-9995-85b08d340b61 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:11.992221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3ad4-8571-41e0-9995-85b08d340b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:11.992515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3ad4-8571-41e0-9995-85b08d340b61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:11.994403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.005046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3ad4-8571-41e0-9995-85b08d340b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-142b3ad4-8571-41e0-9995-85b08d340b61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:12.011765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:12.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:12.014504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 280/558] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:12.016656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.017332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.017860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.039278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:12.041237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:12.100699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598be71d-bdc6-45e1-b1ef-7499b9c5c954 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.120850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 281/559] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:12.121799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.145031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.145656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.146081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.146424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.156516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.158068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:12.173533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:12.174926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:12.175846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:12.182842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 279/560] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:12.183969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.196594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.243121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b60ec1-0fcf-4f39-9103-758adb75812c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.278613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.285594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.286286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.286640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.286981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.287769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:12.288143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.290086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.302223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 282/561] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:12.303153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.304485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a612e-4519-42ce-9906-1cef3ac77578 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.343072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.343730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.344024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.344317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.354758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:12.361510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:12.362529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 280/562] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:12.363386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.425823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d7501-d1b7-4510-ba25-bb1ef9e83ae5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.439966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.461244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.462082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.462082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.462368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.463047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:12.463341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.464788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753fe5d-fbd3-444d-8fbe-a4a6adb1dddd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.467462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86339-1e82-498b-8dfd-ae52ae053bf9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.476771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 283/563] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:12.477942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8f52-c87c-45ab-accb-5c8909be2d9e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.478221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8f52-c87c-45ab-accb-5c8909be2d9e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.478490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8f52-c87c-45ab-accb-5c8909be2d9e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.479840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 284/564] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:12.480693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8f52-c87c-45ab-accb-5c8909be2d9e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.506634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.507194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.507466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.507730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.516517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.518750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.520306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:12.535045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:12.535989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:12.537336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:12.537336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:12.537336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:12.537336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 281/565] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:12.537336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.587804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8f52-c87c-45ab-accb-5c8909be2d9e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.620929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.621517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.621790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.622305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.623626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:12.623929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.625343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.633119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 285/566] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:12.633728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.645756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425f665-8c44-40b2-864d-e3a2c51860b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.666934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.679711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db6f52-e001-46b2-b65d-527722407c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.680353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db6f52-e001-46b2-b65d-527722407c85 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.680633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db6f52-e001-46b2-b65d-527722407c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.680902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db6f52-e001-46b2-b65d-527722407c85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.691409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db6f52-e001-46b2-b65d-527722407c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e6db6f52-e001-46b2-b65d-527722407c85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:12.696527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:12.697495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:12.698337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 282/567] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:12.699022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.699022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.699250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.716653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:12.717880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:12.733802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fb4c4-ba30-474f-87e1-97de853b01da None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.764980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.766992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.767595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.767877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.768181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.768915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=37ca19cdbc204ba995b2d8d46a914fc3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:12.769223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.770646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.776068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 286/568] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/role_assignments?user.id=37ca19cdbc204ba995b2d8d46a914fc3&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:12.776885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.798892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 283/569] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:12.800005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.911839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:12.915753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4ab7-6077-4c70-8ef3-3c148be52776 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.928827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b64ccf-d555-4b53-b15b-c7f957727a2a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:12.955706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.956818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.957394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.957866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.958703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:12.959226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.960970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.968830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:12.969812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:12.969812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:12.970013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:12.975543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 287/570] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:12.976661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:12.981613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:12.983819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:12.999726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:13.000717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:13.001607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:13.001607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:13.001607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:13.004473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 284/571] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:13.005378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.045365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.093557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611729e7-822d-49d4-9ce9-bca9b5676e25 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.115967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729419-e977-473f-b344-d03fd6e6bfab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.130110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.130713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.131225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.131225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.131991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:13.132229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.133596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.142008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 288/572] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:13.142818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.158719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.159352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.159633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.159957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.167813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.171677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.172940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:13.184040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:13.185026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:13.186002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:13.190064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 285/573] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:13.190933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.264278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03767bfe-16ce-4c79-9132-12a8cfea30e9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.297657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.302320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.302931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.303215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.303489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.304136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:13.304495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.305505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9b7f3-3cf9-47ba-b815-c66fc7c6a12f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.305905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.319054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 289/574] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:13.319887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.346613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.347232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.347514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.347789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.358360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:13.363434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:13.364383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:13.365243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 286/575] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:13.366541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.410931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.430849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aff16c-4c4e-434f-b071-9de7ac074bff None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.469649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.470270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.470547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.470819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.477830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c074619-7ee9-4105-af9b-43997ed1e999 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.481143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.482881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:13.497609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:13.498845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:13.499747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:13.499747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:13.499747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:13.499747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 290/576] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:13.499982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.518830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.519450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.519714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.519983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.520741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:13.521045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.522496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.541018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 287/577] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:13.541630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.557161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.614453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b7d6-24ac-452d-a735-242c9a4411b8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.650195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7a1ae-5581-45b3-8e89-f8d9b0f51b9b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.653156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.653559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.653843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.654746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.656879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.664738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.666976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:13.680682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:13.681829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 291/578] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:13.682777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.690997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.691605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.691884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.692238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.693237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:13.693237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.694608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.708102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 288/579] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:13.708102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.785791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.799184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3019a93-ca70-4026-bc6e-d6f271f7c3b7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.813330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d788-f98e-4444-84e1-b66be824ecf9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.836776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.837722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.838202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.838482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.839150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:13.839452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.841043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.850660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:13.851290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:13.851575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:13.851846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:13.855648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 292/580] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:13.856699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.864965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:13.866426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:13.882862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:13.883828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:13.884774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:13.884774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:13.884774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:13.884774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 289/581] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:13.884774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:13.924524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:13.974903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26011f6-1dca-4c2e-8d36-7211889f169a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:13.994089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b09f-b3e4-4061-be65-a392f1ceb64c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.010557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.011159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.011437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.011705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.012464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:14.012764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.014906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.029122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.041074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 293/582] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:14.041074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.043114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.043737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.044012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.044305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.055841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.057334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:14.072130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:14.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:14.073965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:14.073965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:14.073965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:14.073965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 290/583] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:14.073965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.147057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d480e-7fd0-4294-afa2-22ffeb1720a3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.161776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.186131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.186756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.187039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.187312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.187412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937f46e-bc30-499c-a571-fb25eed7b371 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.195891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.197723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:14.213032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:14.213982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:14.214925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 294/584] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:14.216105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.229448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.230058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.230331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.230636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.231248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:14.231554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.233058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.244973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 291/585] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:14.245786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.297123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.335171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df416edd-7a11-488c-a54f-40d963b784d7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.368715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12b258-18b7-4ed4-9387-1c10106e0f89 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.371079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.371710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.372275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.372560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.384390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.385846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:14.400844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:14.401770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:14.402681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:14.402681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:14.402681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:14.402681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 295/586] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:14.402681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.414479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.415369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.415826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.416283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.417163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:14.417471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.418889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.431680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 292/587] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:14.432607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.437280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.518912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c6532-d50b-4fe7-923b-84461eaff83a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.534179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.539478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e9360-2cd2-4932-9e42-8297f34648c3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.556437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.557051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.557337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.557609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.558363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:14.558645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.560085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.575382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.576023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.576337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.576611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.582527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 296/588] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:14.582527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.588365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.589763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:14.602271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:14.603175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 293/589] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:14.604676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.676616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.695466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58a6ea-1c76-4762-9ba3-ac6bc554c955 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.713418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fde416-fc73-472d-8f7b-3b3d5d1f456d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.729876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.730467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.730740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.731005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.731626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:14.731922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.733876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.744852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 297/590] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:14.745917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.753617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.754245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.754525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.754839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.762522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.764623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:14.775394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:14.781664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 294/591] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:14.782858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.795959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.852061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4863b55-f816-4661-ae7c-ceaa54284dbd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.877089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460de4e-4a8f-4ce7-9002-a1dd31512856 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:14.884792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.885412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.885701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.885972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.897020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.898455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.900181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:14.911034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:14.911623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.911902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.912174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.912882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:14.913185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ac1a9c32-8b4b-464f-abbb-b38168fe7ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:14.913750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:14.914813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:14.915718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:14.915718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:14.915718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:14.915887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:14.915956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 298/592] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:14.916334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967706cf-78e0-4019-83d9-2c21d89d8717 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:14.916624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967706cf-78e0-4019-83d9-2c21d89d8717 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:14.916900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967706cf-78e0-4019-83d9-2c21d89d8717 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:14.922726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 299/593] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:14.922726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967706cf-78e0-4019-83d9-2c21d89d8717 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:14.926906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 295/594] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:14.927589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.006889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.028982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967706cf-78e0-4019-83d9-2c21d89d8717 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.031477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a239-36c6-433c-98dc-538161b44510 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.063252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.063868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.064141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.064442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.064975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.065574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.065853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.066127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.066856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:15.067167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.068639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732584-9bdb-4c5f-94e3-52b073294c64 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.071185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.072642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:15.084157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:15.085230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 300/595] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:15.086131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.086676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 296/596] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Aug 18 11:03:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:15.087677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.087976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.088252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.106288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:15.107470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:15.144707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.174141 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 297/597] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:15.174944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.190693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a4f53-b830-40d8-a6e6-10366917def8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.225933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.226531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.226863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.227140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.227775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:15.228076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.229472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.241159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 301/598] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:15.241953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.257941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.298215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fa40f-7749-4c18-a4af-94d7573235ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.333721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.334474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.334809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.335140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/aefa27e6907c4ba28483d31a68753c31/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.350142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=aefa27e6907c4ba28483d31a68753c31, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.351665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.363548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae73ad3-6357-4492-8da0-cb35988b93c4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.372579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 298/599] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:15 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/aefa27e6907c4ba28483d31a68753c31/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:15.372579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.380530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.403702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.405076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.405352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.405621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.406588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:15.407072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.408780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.425426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 302/600] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:15.426474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.518445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2541d-a4dd-4263-aa64-fdd081cc68a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.539838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.550231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3849b-59da-4e60-9b70-91b4141df408 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.564768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.565864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.566460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.566803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.577618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.579262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.587061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.587696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.587969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.588261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:15.594328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:15.595269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:15.596787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:15.596787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:15.596787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:15.596787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 299/601] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:15.596787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.599821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.601308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:15.613771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:15.614806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:15.615691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 303/602] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:15.616170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.656567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.705703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b8661-7002-40ab-bef7-0d42803a555f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.728144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1101e7d-ee38-4700-bdb7-72e1537840cc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.743237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.743896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.744173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.744473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/5e78415ea98940bbbc59c733c8ffefa4/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.760933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=5e78415ea98940bbbc59c733c8ffefa4, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.762324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.762449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.762946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.763221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.764041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.776131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:15.781486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:15.782483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:15.783368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:15.784708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 300/603] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:15 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/5e78415ea98940bbbc59c733c8ffefa4/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:15.786768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.787600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 304/604] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:15.788742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.814882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.920487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:15.929348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e103daf-074d-453f-b7bb-dae907902094 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.948546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4e8e4-7041-4cfa-8f4f-0570702f8d30 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:15.968856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.969622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.970002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.970375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.971167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:15.971576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:15.973610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:15.986886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 305/605] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:15.987845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:15.995239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:15.996354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:15.998020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:15.998472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:15.999277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:15.999740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.001350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.018789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 301/606] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:16.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.110201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800d1a2-d17e-48ea-8722-72c6531c0ce7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.139579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:16.150997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.152357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.153089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.153518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.154854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:16.155563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.157854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.165674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c242fc-2e69-406f-8dcc-b172f9a51c86 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.169830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 306/607] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:16.171069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.192599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:16.212789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.213979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.214408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.215711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/1f43ff51c5f84d34978c46bce6db67ee/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.234377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=1f43ff51c5f84d34978c46bce6db67ee, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.236131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.260434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 302/608] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:16 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/1f43ff51c5f84d34978c46bce6db67ee/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:16.261473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.301281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95aee73-0cf9-4339-9589-5ebde830008f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.343793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.344744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.345224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.345779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.354311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.356241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:16.369923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:16.371537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:16.373480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:16.375578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 307/609] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:16.381836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.470497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:16.496164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44e82a-5403-447d-99c1-829f6170cde2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.502513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2e969-d6d9-43a6-8a9c-4a3957be9e43 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.533268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.534151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.534574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.535020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.535348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:16.536181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:16.536834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.538637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.548570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.549298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.549680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.550054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.555101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 308/610] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:16.561738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.566371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.568402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:16.588608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:16.589792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:16.590912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:16.590912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:16.590912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:16.592959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 303/611] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:16.600282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.675594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636caea-0b9a-4a97-a3bd-c204ec4dfd37 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.712806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.713796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.714218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.714635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/a09c08eca5be4a1b8977ff91bb468358/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.719209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:16.731835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=a09c08eca5be4a1b8977ff91bb468358, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.733365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.738979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a775f0-470c-4408-b642-ad48e6edecf0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.762062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 309/612] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:16 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/a09c08eca5be4a1b8977ff91bb468358/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:03:16.785297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.786420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.786803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.787180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:16.788504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:16.789095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:16.790629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:16.806210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 304/613] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:16.812961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:16.955254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f094ff-ea8b-4699-8299-1f63feb6832d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:16.994059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:16.994967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:16.995391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:16.995803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.005063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:17.006800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:17.020744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:17.021926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:17.023084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:17.025348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 310/614] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:17.033923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:17.088957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:17.164963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573770f-c145-4651-91c0-992c04316d38 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:17.210729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:17.211342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:17.211685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:17.211964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.212736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:17.213044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:17.214903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:17.237845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 305/615] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Aug 18 11:03:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:17.243265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:17.312267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:17.387460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7788163-9c57-437c-b303-0be8d3bf943d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:17.427666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:17.428342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:17.428624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:17.428896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.446227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:17.447682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:17.467422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:17.468464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:17.469359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:17.469359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:17.469359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:17.469359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 311/616] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:17.474162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:17.600713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:17.608247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c045f-7472-4aff-806d-f1bfe8c7243e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:17.655684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:17.656474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:17.656931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:17.657335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.658084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:17.658490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:17.659973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c7bd-1e81-4466-828d-16170ea8ce8e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:17.672122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 306/617] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:17.677361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:17.798294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c62a54-bce7-4283-a7db-05c5138169b2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:17.798745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c62a54-bce7-4283-a7db-05c5138169b2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:17.799125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c62a54-bce7-4283-a7db-05c5138169b2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.801079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 307/618] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:17.806040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:17.806328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:17.806601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.807082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:17.810218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf975-f5e2-4cfa-8d47-3d9c14c52649 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:17.851308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:17.852464 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:17.856324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:17.856955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:17.857844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:17.858115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:17.870720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:17.872460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:17.891765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:17.892860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:17.894078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:17.895770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 312/619] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:17.902410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:17.987304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 308/620] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:18.049046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d1284-0019-45cd-8479-3251b91736f8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:18.084524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:18.085129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.085406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.085674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.086362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:18.086665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:18.088081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:18.101727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 313/621] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:18.108046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:18.217552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:18.236383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f5a2-e369-4ccb-92ad-e689e63e57c0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:18.280870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00f27d-04d9-41cc-b09d-8b9547dcb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:18.281534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00f27d-04d9-41cc-b09d-8b9547dcb496 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.281812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00f27d-04d9-41cc-b09d-8b9547dcb496 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.282085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00f27d-04d9-41cc-b09d-8b9547dcb496 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/37ca19cdbc204ba995b2d8d46a914fc3/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.305202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00f27d-04d9-41cc-b09d-8b9547dcb496 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=37ca19cdbc204ba995b2d8d46a914fc3, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:18.306685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00f27d-04d9-41cc-b09d-8b9547dcb496 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:18.313558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:18.325774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 309/622] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:18 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/37ca19cdbc204ba995b2d8d46a914fc3/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:18.351813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:18.392066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282aa5-2fae-40c2-a53f-d34a453ceb4a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.392485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282aa5-2fae-40c2-a53f-d34a453ceb4a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.392660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282aa5-2fae-40c2-a53f-d34a453ceb4a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.394284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 310/623] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:18.402479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.402798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.403022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.440712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:18.441938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:18.470785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83733c-b910-4ad3-a52a-d0c4ff716ac6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:18.518911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:18.520225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.520910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.521383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.537192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:18.543921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:18.545385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:18.546399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:18.552713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 314/624] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:18.556026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:18.579278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 311/625] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:18.672794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:18.705367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b90a-d52f-4e58-a7d1-a7ef27550e12 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:18.752899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:18.753514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.753799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.754067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.754736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:18.755039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:18.756445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:18.765740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 315/626] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:18.771611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:18.862851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:18.902593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d81ebd-04df-4ecd-8e7c-a872ad1c57d8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:18.907987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:18.940503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:18.941475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:18.942021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:18.942519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:18.953831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:18.955295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:18.973876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:18.974871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:18.975787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:18.975787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:18.975787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:18.975787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 312/627] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:18.981159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.058814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4f8ba-486e-44e7-ae80-3345f49e1f8c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.106364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.107024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.107322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.107581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.126777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:19.135796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:19.136769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:19.137614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 316/628] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:19.140508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:19.145344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.165848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efa67f-8d8f-4608-af1c-f763097bb42a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.190956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:19.208033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.209090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.209513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.210065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.210850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:19.211345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:19.213065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfeda4-3f70-474e-b57c-5ea5609ddf7c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:19.231923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 313/629] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:19.234105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de3cca-4119-4d94-8016-801c46e940e6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.234698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de3cca-4119-4d94-8016-801c46e940e6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.235161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de3cca-4119-4d94-8016-801c46e940e6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.237082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 314/630] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:19.238161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de3cca-4119-4d94-8016-801c46e940e6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.282041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa559ad4-ea1e-4385-a406-29d601a200d4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.322924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.323909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.324442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.324924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.326360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:19.327143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:19.328756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84793c58-0aa2-4a77-9a4e-cd39e17239b7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:19.342723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 317/631] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:19.344318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.344817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.345321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.363202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de3cca-4119-4d94-8016-801c46e940e6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.364529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:19.365760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:19.405636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.406268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.406572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.406816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.415196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:19.416661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:19.424948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c10ebafb-527f-4ebe-b773-a7ed8c421599 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:19.429440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:19.430442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:19.431423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 315/632] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:19.432012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dad5bb-7a08-4df7-928c-395b0e50651e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.432012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dad5bb-7a08-4df7-928c-395b0e50651e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.432673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dad5bb-7a08-4df7-928c-395b0e50651e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.433691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 316/633] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:19.434280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dad5bb-7a08-4df7-928c-395b0e50651e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.451849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 318/634] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:19.451849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.485846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:19.547940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dad5bb-7a08-4df7-928c-395b0e50651e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.582514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e7a1-ec6d-473b-b836-8245bf153e15 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.583762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45147216-acce-4890-b673-c635a8f17134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.584483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45147216-acce-4890-b673-c635a8f17134 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.584823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45147216-acce-4890-b673-c635a8f17134 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.585206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45147216-acce-4890-b673-c635a8f17134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.597566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45147216-acce-4890-b673-c635a8f17134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:19.599493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45147216-acce-4890-b673-c635a8f17134 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-45147216-acce-4890-b673-c635a8f17134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:19.617834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:19.619060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:19.620157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:19.620157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:19.620157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:19.627020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 317/635] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:19.629066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834bf01-f394-46e6-937f-492331082717 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.629592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834bf01-f394-46e6-937f-492331082717 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.630198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834bf01-f394-46e6-937f-492331082717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.630919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.631732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.632165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.632686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.633608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:19.634060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:19.635666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:19.657339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a834bf01-f394-46e6-937f-492331082717 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:19.658671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a834bf01-f394-46e6-937f-492331082717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:19.659823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 319/636] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:19.660760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.717216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:19.769091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 318/637] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:19.770094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4eed-d8b3-4313-a53c-c26dbc055236 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.770615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834bf01-f394-46e6-937f-492331082717 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.804589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:19.820699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.821302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.821576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.821844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:19.822485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:19.822784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:19.828537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:19.845877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 320/638] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:19.847046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:19.907194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834bf01-f394-46e6-937f-492331082717 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:19.950629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:19.952585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:19.953056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:19.953746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.000744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda9dc4-a6eb-40ca-9fbc-0003c2ca6f90 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.012013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.038314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.039220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.039731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.040367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.052004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:20.057443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:20.058412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:20.059267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:20.059907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.061020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.062377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 321/639] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:20.063515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.095558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.188184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac1d9-96c7-45e7-83c9-0165210ded6f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.196032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 319/640] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:20.196032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.231844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.233126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.233413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.233685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.242016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.243542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:20.255996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:20.257366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 322/641] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:20.258560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.292411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.312834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42947bd5-f9e8-472a-986d-63e8d630b29b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.337414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.352333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.353045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.353330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.353602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.364743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:20.367745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eef6e-b4ab-4da7-b373-bbd1ca0e074a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:20.370218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:20.371256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:20.372411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 320/642] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:20.373748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.413971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.414940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.415429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.415701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.416411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:20.416722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.418102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.431512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 323/643] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:20.431512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.490669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e660b-47c2-4342-ad8a-5503f405bd90 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.528058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.529692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.530566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.531279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.531708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.537286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e859f-a73a-41e7-a47f-05acac5ab4eb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.542482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.544553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:20.559609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:20.561543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:20.562672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:20.562672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:20.562672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:20.562672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 321/644] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:20.562933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.572881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.573514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.573829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.574101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.575040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:20.575380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.577514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.596734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 324/645] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:20.598428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.603325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.673926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b48b8e-c89d-46a8-b69a-a4a6e783bf5f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.700487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6b1c9-0ac6-4e37-9ddb-4457f4222030 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.707668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.708292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.708577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.708849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.709507 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:20.709810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.711180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d4137-cb3a-4e7b-b897-aa64a8580e2a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.719466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 322/646] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:20.720449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad50ee-7910-4ce7-a8c9-7fb4898b8183 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.720732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad50ee-7910-4ce7-a8c9-7fb4898b8183 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.721004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad50ee-7910-4ce7-a8c9-7fb4898b8183 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.722204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 323/647] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:20.722938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad50ee-7910-4ce7-a8c9-7fb4898b8183 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.738450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.739071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.739343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.739610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.751249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.752814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:20.771942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:20.773152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:20.774033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:20.774033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:20.774033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:20.774033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 325/648] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:20.774033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.776842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.830559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.838385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad50ee-7910-4ce7-a8c9-7fb4898b8183 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.871444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.872037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.872658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.873430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.874073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:20.874374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.875752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.887743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 324/649] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:20.888583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:20.888848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fac36b-81ed-46aa-b7e6-8e1fb661a5c7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:20.922956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:20.923558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:20.923837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:20.924109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:20.990910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:20.992390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:20.994179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:20.994847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aa6f9-d5f3-4ee7-a9b5-9fa40d8e522f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.029708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.030695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.030695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.030918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.041053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.042498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-783d8382-f174-4842-9faf-4cceb5a54ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:21.057555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:21.058515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:21.059390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:21.059390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:21.059390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:21.059390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 325/650] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:21.059971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.060164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.061062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.078677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:21.079887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:21.082027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.111858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 326/651] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:21.112762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.151003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 326/652] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:21.152036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.218434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616355c5-0644-4105-bf9d-19e353d1c326 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.220541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.259559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.260155 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.260479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.260752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.261388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:21.261686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.263080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.272058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd3e80-62a5-427f-a661-c6b24afd716c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.275752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 327/653] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:21.280870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.312155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.312807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.313089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.313358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.320548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.322242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.324047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:21.336421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:21.337551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:21.338658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:21.339200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 327/654] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:21.339929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.401031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63383eca-13dc-4c5e-ac4d-cea757b80f29 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.438275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.438985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.439324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.439640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.440318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:21.440656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.442030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.455194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 328/655] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:21.456004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.472798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0def619-ea9e-47d5-82d0-4201e75a42ba None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.483436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.512861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.514115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.514399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.514996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.526500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.527948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:21.543043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:21.544064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:21.545142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:21.545142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:21.545142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:21.545142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 328/656] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:21.545142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.573050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc7221-4652-42d5-8756-b0ace66122ce None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.605731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.608634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.609452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.609613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.609804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.619232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.621561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:21.636470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:21.637603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:21.639360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 329/657] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:21.640279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.664751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b074b-86bf-4c42-a6ee-fbd5bdd9ac12 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.699366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.699957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.700266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.700543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.701849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:21.702159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.703585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.715462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 329/658] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:21.716380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.733262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.753221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d678fa-d8a5-49e0-883d-bb096e36aaff None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.787941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.788595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.788877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.789150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.796676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.798117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:21.809099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:21.810272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 11:03:21.811433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:21.812326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 330/659] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:21.813282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.859190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0817-30e7-4e2a-a17d-fb68a841866d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.870016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:21.895000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.895601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.895882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.896151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.896812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:21.897110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:21.898468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:21.909891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 330/660] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:21.910687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.916040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f7e0-2853-4296-bc0e-b8aaee1494f7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:21.950060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:21.950696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:21.951513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:21.951785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:21.961656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:21.966820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:21.967930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 331/661] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:21.968796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:21.971878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.026126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc98eb-6e56-4b26-a2ec-a2ce0932d0fa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.059212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.059810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.060641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.061036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.061990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:22.062290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.063702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.081303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 331/662] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:22.082188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.082962 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892d8e4-18ed-4202-b3cb-59291a523dd4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.111417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.118900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.119501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.119781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.120051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.120805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=aefa27e6907c4ba28483d31a68753c31) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:22.121115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.122856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.128698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 332/663] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:21 2022] GET /identity/v3/role_assignments?user.id=aefa27e6907c4ba28483d31a68753c31&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:22.129476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.193294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7f581-386f-48bf-9111-1a10199cda37 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.206509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.239653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.240668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.241017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.241355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.242158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:22.242527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.244034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.245630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512dfe9-1075-4656-a154-5a040b864c72 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.269660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 332/664] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:22.269660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.292222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.292854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.293128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.294027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.305317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.306783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:22.319083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:22.320073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:22.321154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 333/665] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:22.321691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.380604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.397998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df4fd9-100c-4dd3-8fe5-77d25440af01 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.438593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8ec0a-d2c0-49ab-8be0-41c1bf00acd1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.444120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.444991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.445490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.445966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.447173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:22.447764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.449786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.458799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 333/666] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:22.459758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.472930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.484155 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.485041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.485449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.485844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.561671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.563523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.588346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2cd64-d861-49cd-8d1b-275526002e41 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.625021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.625695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.625880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.626148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.644954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.683567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 334/667] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:22.683567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.709381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.710930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.711634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.791754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406ce0e-fdac-4d9c-810d-bc4fc0046d24 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.820791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 334/668] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:22.821726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.834297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.834998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.835342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.835681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.836472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:22.836840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.838265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:22.857092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 335/669] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:22.857092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:22.885403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.940800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1754b2-23d4-4fd4-a73a-1c5a19745361 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.955385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:22.972136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa767d-c2a6-4548-8a51-c7e4d800ff9c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:22.977484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:22.978110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:22.978384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:22.978650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:22.991170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:22.992934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:23.008519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:23.009688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:23.010753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:23.010753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:23.010753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:23.011985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 335/670] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:23.012809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.016894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.017539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.017821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.018095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.030246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.031955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:23.046587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:23.047714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:23.048651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:23.048651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:23.048651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:23.048651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 336/671] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:23.048651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.130963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:23.134013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757b766-1721-4c22-a972-ce63cdd1dc8f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.162350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016b2f-fba9-4a0e-bb1d-2727e6357771 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.171007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.171789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.172158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.172517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.184176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.186042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.198449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.199307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.199732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.200160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:23.201183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:23.202342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:23.203284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:23.203284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:23.203284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:23.203284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 336/672] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:23.203510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.204221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=5e78415ea98940bbbc59c733c8ffefa4) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:23.204550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.206025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.214017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 337/673] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/role_assignments?user.id=5e78415ea98940bbbc59c733c8ffefa4&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:23.214851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.224020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:23.374354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31723d4f-222e-47d6-9f4a-390e534f3dbb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.378090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a2819-52ff-407d-8040-6cd77a608a8b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.433029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.433801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.434082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.434357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.435085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:23.435362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.436772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.439150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.439913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.440413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.440815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.441578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:23.442017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.443515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.453391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 337/674] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:23.454000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.463441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 338/675] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:23.463626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.472514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:23.540182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:23.585299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63856b-3667-4e22-baed-d2b7fdd05ce5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.593472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb2eb9-0bdf-4e7c-b906-961c3c62cf61 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.629913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.631963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.632747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.633765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.635341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:23.636291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.636676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.637186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.637555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.637933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.641256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.646959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.648558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.659788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 338/676] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:23.661657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:23.662044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:23.663485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:23.664449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 339/677] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:23.665084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.747360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:23.773553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53468def-aa60-40cf-915e-7f63a1506d4c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.784232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f8fe-64a6-466b-bdcb-d4ecf3e4fa05 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.800733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:23.815686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.816450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.816795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.817190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.824908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.827288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.828447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:23.829582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:23.830043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:23.831153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:23.839222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:23.840843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:23.842653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:23.843666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 340/678] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:23.844163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.844856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:23.860018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:23.861020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:23.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:23.869022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 339/679] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:23.869168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:23.980999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5d231-38d8-46b9-8590-077737d8e5bb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:23.997011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec335d-29ca-4353-a01d-900e84f1f844 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.026774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.029686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.029968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.030246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.031086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:24.031467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.032938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.037009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:24.043924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.044977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.045459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.045926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.046815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:24.047318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.049294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.052080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 341/680] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:24.053050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.064284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 340/681] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:24.065640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.098007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:24.165226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba081b-1fda-4782-8cb3-f10f09d7e34b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.183731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c231c-eafa-4797-bc86-16fafd680cee None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.205225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.205824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.206097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.206374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.207068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:24.207375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.208834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.225292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 342/682] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:24.225292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.229805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704197b-a785-4206-9a49-c0d080a747d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.230445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704197b-a785-4206-9a49-c0d080a747d0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.230724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704197b-a785-4206-9a49-c0d080a747d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.230997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704197b-a785-4206-9a49-c0d080a747d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.295878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:24.309073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704197b-a785-4206-9a49-c0d080a747d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.311066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704197b-a785-4206-9a49-c0d080a747d0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.328452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:24.332060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a05ee-28a2-40a9-8fed-d65b0fa412c1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.370923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.371563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.371840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.372391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.373742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=a09c08eca5be4a1b8977ff91bb468358) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:24.374049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.375527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.392221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 343/683] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/role_assignments?user.id=a09c08eca5be4a1b8977ff91bb468358&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:24.392221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.446288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 341/684] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:24.447735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dce586-783c-410a-ab27-c7521b6044bc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.448024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dce586-783c-410a-ab27-c7521b6044bc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.448330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dce586-783c-410a-ab27-c7521b6044bc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.449699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 342/685] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:24.453704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dce586-783c-410a-ab27-c7521b6044bc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.516891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d73e4-75d6-4a42-bd8c-d3b8c5a24cc4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.563251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.563859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.564215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.564619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.565382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=1f43ff51c5f84d34978c46bce6db67ee) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:24.565692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.568072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b43b38-2021-4ef7-ae9f-9322cfdac4d0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.575834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 344/686] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/role_assignments?user.id=1f43ff51c5f84d34978c46bce6db67ee&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:24.577408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.577697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.577977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.588401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:24.597936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:24.599237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:24.603180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dce586-783c-410a-ab27-c7521b6044bc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.631957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:24.645963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.646702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.647057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.647403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.660394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.662326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:24.683227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:24.684350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:24.685824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:24.685824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:24.685824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:24.688370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 343/687] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:24.691162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 345/688] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:24.691291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.815003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b1500-e238-41b3-91e4-d1f7831a00ad None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:24.851564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:24.852168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:24.852478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:24.852752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:24.853399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:24.853760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:24.855131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:24.868199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 344/689] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:24.874307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:24.981065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfc403-3fef-4353-b554-9e111765198f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:25.020609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:25.021236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.021513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.021792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.029373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:25.030216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:25.034273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:25.039316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:25.047431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:25.048558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:25.049462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 346/690] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:25.055959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:25.086097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:25.177374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ec234-7d12-4d51-82a0-0f101c88161d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:25.195899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0017b4-b15c-47f9-b766-f8fe423dd9c2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:25.217432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:25.218066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.218406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.218684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.219276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:25.220707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:25.231781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:25.234246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:25.236525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:25.238744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 345/691] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Aug 18 11:03:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:25.239591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:25.252108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:25.252743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.253014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.253279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.253976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:25.254274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:25.255697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:25.277717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 347/692] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:25.284180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:25.324831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:25.396305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b939025-89a3-429e-b2db-73dadc670c0c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:25.452588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:25.470863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:25.471473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.471745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.472009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.472666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:25.472964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:25.474341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:25.489010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 346/693] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Aug 18 11:03:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:25.489932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:25.508675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f031c20-cd9f-480a-a411-3dc3483c08cb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:25.554742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:25.555720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.556093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.556500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.557317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=37ca19cdbc204ba995b2d8d46a914fc3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:25.557725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:25.559240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efe617-2c67-4eb1-be5a-c65ed17370d8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:25.567458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 348/694] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:25 2022] GET /identity/v3/role_assignments?user.id=37ca19cdbc204ba995b2d8d46a914fc3&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:25.569589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8661-be6b-44c7-a17e-172c09ca27b2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.570004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8661-be6b-44c7-a17e-172c09ca27b2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.570377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8661-be6b-44c7-a17e-172c09ca27b2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.576323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 349/695] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:25.578784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.579069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.579342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.602006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:25.603498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:25.689242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b9826-59de-4cf9-9f44-909e110adcd5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:25.740240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:25.743194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 350/696] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 11:03:25.748803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:25.748803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:25.748803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:25.749103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:25.749555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:25.749850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:25.751209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:25.757906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 347/697] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Aug 18 11:03:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:26.038747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:26.113776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:26.146394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e960852-8d48-429c-8761-77b9ba42fa74 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:26.189890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:26.190534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:26.190809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:26.191075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:26.202852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:26.204471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:26.232026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:26.233089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:26.235058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:26.235058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:26.235058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:26.235058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 351/698] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:26.239133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:26.377114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755f61-a8ac-4a5f-bfd4-eacc371b2d22 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:26.429540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:26.430536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:26.431024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:26.431502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:26.432385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:26.432906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:26.434474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c0442-cfbd-42ca-ae36-b1172759aa04 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:26.454308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 348/699] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:26.460121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:26.612812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf0df7-590d-4c7b-8b28-6605cf9e0317 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:26.645689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:26.659651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a497967-a04e-4056-aa51-9bc21b3f2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:26.660284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a497967-a04e-4056-aa51-9bc21b3f2691 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:26.660565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a497967-a04e-4056-aa51-9bc21b3f2691 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:26.660831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a497967-a04e-4056-aa51-9bc21b3f2691 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:26.661464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a497967-a04e-4056-aa51-9bc21b3f2691 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:26.662880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a497967-a04e-4056-aa51-9bc21b3f2691 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:26.698440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 352/700] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:03:26 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:26.804895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58439332-2d48-4a68-8525-24a5e48e0d27 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:26.805896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58439332-2d48-4a68-8525-24a5e48e0d27 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:26.806409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58439332-2d48-4a68-8525-24a5e48e0d27 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:26.808431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 349/701] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:26.814515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d76da6-33c3-4ec9-ae0d-27d6301baa38 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:26.814829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d76da6-33c3-4ec9-ae0d-27d6301baa38 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:26.815067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d76da6-33c3-4ec9-ae0d-27d6301baa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:26.837596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-08d76da6-33c3-4ec9-ae0d-27d6301baa38 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:26.838821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-08d76da6-33c3-4ec9-ae0d-27d6301baa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:26.931445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 353/702] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:27.184388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4093c8-ea8f-4209-a3bf-d4968c7ff98c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:27.185576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4093c8-ea8f-4209-a3bf-d4968c7ff98c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:27.186085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4093c8-ea8f-4209-a3bf-d4968c7ff98c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:27.187928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 350/703] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:27.195510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:27.196433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:27.196914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:27.215153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:27.217485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:27.222991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4093c8-ea8f-4209-a3bf-d4968c7ff98c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:27.340897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:27.346266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 354/704] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:27.401025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4093c8-ea8f-4209-a3bf-d4968c7ff98c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:27.453018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:27.453561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:27.453834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:27.454102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:27.454729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:27.456097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:27.468808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:27.469822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:27.471258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 351/705] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:27 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:27.475969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:27.532133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:27.612673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02859b-a385-425a-b36f-08b8bd219cf8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:27.655615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:27.656823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:27.656823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:27.656823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:27.660213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] RBAC: Adding query filter params (name=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:27.660213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:27.660213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:27.674559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 355/706] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:27.685585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:27.733495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:27.822111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ebf67-1f3f-4860-b076-c80ecae9da9d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:27.843524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:27.860354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:27.860598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:27.860847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:27.861047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:27.864231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:27.864231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:27.864231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:27.880933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 352/707] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:27.921749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854c684-d77e-4412-957d-640f09bbcefe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:27.924734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:27.985025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65337c82-2223-4108-8ad7-2dbd63cf9e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:27.985645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65337c82-2223-4108-8ad7-2dbd63cf9e91 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:27.985926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65337c82-2223-4108-8ad7-2dbd63cf9e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:27.986194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65337c82-2223-4108-8ad7-2dbd63cf9e91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.009963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65337c82-2223-4108-8ad7-2dbd63cf9e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-65337c82-2223-4108-8ad7-2dbd63cf9e91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:28.015589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:28.016658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:28.017960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 356/708] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:28.023163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:28.075096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00d509-e0f0-4e28-9154-e0edb87004a7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.076526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00d509-e0f0-4e28-9154-e0edb87004a7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.076818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00d509-e0f0-4e28-9154-e0edb87004a7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.079241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 357/709] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:28.085082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.086198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.086964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.112051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:28.116160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:28.230432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 358/710] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:28.244288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae2d32-3b94-450a-888d-1db1c8143d60 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:28.323850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:28.326911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.327198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.327467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.328149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:28.328482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:28.329860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:28.364073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 353/711] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:28.364284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:28.547222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:28.555464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8954967-479d-4407-abdc-747e5e4d4e14 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:28.593943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:28.594588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.594868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.595138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.606712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:28.608238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:28.613594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0bad499f-fb38-421f-b33d-a235aff66c36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:28.625531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:28.626506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:28.627393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:28.627393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:28.627393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:28.627966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 359/712] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:28.630749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff411ca-7a38-4fd7-a682-59b719710ff9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.631037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff411ca-7a38-4fd7-a682-59b719710ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.631515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff411ca-7a38-4fd7-a682-59b719710ff9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.633504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 360/713] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:28.634474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff411ca-7a38-4fd7-a682-59b719710ff9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:28.716747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:28.731986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72e8f-1a18-4344-8eb2-b350b03c4d62 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:28.774301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff411ca-7a38-4fd7-a682-59b719710ff9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:28.780617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb063818-f8ae-455f-b0e7-f5577d3066e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:28.781462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb063818-f8ae-455f-b0e7-f5577d3066e9 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.781946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb063818-f8ae-455f-b0e7-f5577d3066e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.782438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb063818-f8ae-455f-b0e7-f5577d3066e9 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.783696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb063818-f8ae-455f-b0e7-f5577d3066e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:28.785707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb063818-f8ae-455f-b0e7-f5577d3066e9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:28.813206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 354/714] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:03:28 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:03:28.813206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.813206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.813206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.850510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:28.852870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:28.855955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:28.856604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:28.857548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:28.857892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:28.859229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:28.859547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:28.862556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:28.887787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 361/715] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:28.894920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:28.899059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:28.969466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 355/716] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:29.071683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a9f39-7c56-42c2-a977-3a8d94876bd9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:29.109378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:29.110010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.110286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.110558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.118540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:29.119983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:29.131847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:29.132823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:29.133767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 362/717] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:03:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:29.139581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:29.181221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:29.253073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65798c-6549-41be-bb0c-92384f86e8fe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:29.260520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:29.290350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:29.291016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.291302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.291577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.292374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:29.292685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:29.294118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:29.314314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 356/718] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:03:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:29.321063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:29.397753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:29.465978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96314-bec4-4a84-bc9a-7888389d7e51 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:29.506491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:29.519252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39db6d-beaf-490e-9736-0eebae628ca5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:29.530558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad200db-281c-4e25-9491-cf9a771a2ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:29.531245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad200db-281c-4e25-9491-cf9a771a2ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.531518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad200db-281c-4e25-9491-cf9a771a2ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.531788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad200db-281c-4e25-9491-cf9a771a2ac9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.551161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad200db-281c-4e25-9491-cf9a771a2ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4ad200db-281c-4e25-9491-cf9a771a2ac9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:29.559739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:29.560789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:29.561684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:29.564952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 363/719] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:29.566367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796c923-11e9-4a3c-89a6-aa439da2a64a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.566653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796c923-11e9-4a3c-89a6-aa439da2a64a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.566929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796c923-11e9-4a3c-89a6-aa439da2a64a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.569576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 364/720] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:29.571650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796c923-11e9-4a3c-89a6-aa439da2a64a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:29.577816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:29.578675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.579130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.579599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.667052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:29.670339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:29.689801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796c923-11e9-4a3c-89a6-aa439da2a64a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:29.715959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:29.732330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:29.733080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.733364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.733640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.734347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:29.734602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:29.736079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc30ea-bc64-4174-b8f7-59d174bfb141 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:29.750935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 365/721] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:29.752131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.752457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.752734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.772894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:29.774379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:29.789584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:29.804288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 357/722] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:29.804288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:29.868864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 366/723] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:29.869815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:29.927701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c39ac-f15f-40fb-82c5-787169ddd23d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:29.972522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:29.973517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:29.974024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:29.974543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:29.985525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:29.987246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:30.002718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:30.003716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:30.004629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:30.004629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:30.004629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:30.006676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 358/724] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:30.012377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.046012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b59b42-fed2-4fcb-a076-ba8bb6d1788c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.064353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.069002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.104877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.105764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.106225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.106645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.128731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.130422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.142511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ba962-8f28-4b80-9b8c-253b1546d57e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:30.150692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:30.151860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:30.153008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:30.153008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:30.153008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:30.155220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 367/725] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:30.160594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.177126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.177795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.178018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.178294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.178942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:30.179248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.180663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.193879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 359/726] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:30.198926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.294318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff11332-4b6d-48a7-9576-459a9825bd82 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.323400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad126f-4c1a-42d6-bcd7-d01a8d43bed5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.330961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.331560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.331672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.331834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.332109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.332874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:30.333179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.334586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6066e-33a6-4c5e-a1f1-8dfc61d4744a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.352809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 368/727] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:30.354004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e08c4-14b8-41f1-adb9-e824883ac778 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.354289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e08c4-14b8-41f1-adb9-e824883ac778 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.354569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e08c4-14b8-41f1-adb9-e824883ac778 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.359868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 369/728] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:30.360623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e08c4-14b8-41f1-adb9-e824883ac778 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.363051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.363692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.363972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.364273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.371755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.374182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:30.391921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:30.392964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:30.393953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:30.395202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 360/729] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:30.395273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.404233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.487177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e08c4-14b8-41f1-adb9-e824883ac778 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.504599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafc209-b740-4dff-b3fe-225073758957 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.527052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a56720-babc-4390-b96c-4b823c93e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.527683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56720-babc-4390-b96c-4b823c93e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.527965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56720-babc-4390-b96c-4b823c93e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.528265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56720-babc-4390-b96c-4b823c93e2f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.540440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a56720-babc-4390-b96c-4b823c93e2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:30.541487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.542110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.542388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.542655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-57a56720-babc-4390-b96c-4b823c93e2f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:30.545782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:30.546741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:30.547615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 370/730] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:30.548912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.549000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.549216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.549480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.550917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:30.562526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:30.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 361/731] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:30.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.572338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:30.573548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:30.597039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.645435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 371/732] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 11:03:30.652346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.678276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.682844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d8cbe-7da8-4f21-a71e-315b9eea65bc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.731710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.732404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.732734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.733012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.734177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:30.734489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.736220 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.760011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 362/733] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:30.761184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.786418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d57b01-46a1-4567-987b-9db8c55ad4e8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.829510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.830466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.830937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.831419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.832301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:30.832795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.834362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.845556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 372/734] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:30.846650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.858687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.878598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2a272-6f11-427a-b367-54cfee0bced1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.917084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.917713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.917997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.918245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:30.918943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:30.919275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:30.921051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:30.930398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:30.933311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 363/735] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:30.936338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:30.956038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ac701-f98b-4281-8342-0d78d2e523b1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:30.990445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:30.991333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:30.991856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:30.992133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.053930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850013d-3776-4d01-a5be-eed11812bfca None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.070634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.072293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.089965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.090597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.090881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.091687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.103225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.104727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.108490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:31.121148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:31.122135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:31.123346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:31.123346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:31.123346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:31.123346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 364/736] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:31.123715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.164596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:31.189811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 373/737] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 11:03:31.190928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.237913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dc6d5-c9f4-4c1f-a5c0-62c4f022be2f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.272151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.272766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.273041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.273317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.274010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.275385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.289790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 365/738] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:03:31 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:31.290531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.298960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b7dcf-2239-4f05-87a6-5bd89c730f69 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.351435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.352054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.352362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.352636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.353376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=1f43ff51c5f84d34978c46bce6db67ee) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:31.353688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.355175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.361406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 374/739] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/role_assignments?user.id=1f43ff51c5f84d34978c46bce6db67ee&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:31.362271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.378353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:31.423475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:31.446649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0921405-f580-45b7-a7fe-0093f5df3789 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.498447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.499093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.499374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.499648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.501168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:31.501325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.503496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f23235-f7ab-4414-85e6-bd52bc838b6d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.519246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 366/740] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:31.520838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee769-3285-4f5b-a8ea-f7137ae3024e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.521354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee769-3285-4f5b-a8ea-f7137ae3024e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.521777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee769-3285-4f5b-a8ea-f7137ae3024e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.522373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a5f27-4364-4f7d-aad2-8c02b8ce560a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.527492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 367/741] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:31.528664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee769-3285-4f5b-a8ea-f7137ae3024e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.574434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.575714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.576104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.576571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.590878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.594006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0b29dea2-699d-4798-aec7-eaf5fbbea508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:31.613593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:31.614596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:31.615677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:31.615677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:31.615677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:31.615677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 375/742] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:31.616567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.617228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.617507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.642697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:31.644000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:31.648257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee769-3285-4f5b-a8ea-f7137ae3024e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.667293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:31.686689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.687668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.688176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.688708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.696803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.698549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:31.711282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:31.712312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:31.713264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:31.715804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 368/743] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:31.717864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.728224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 376/744] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:31.728224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.748205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:31.845074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebf835-fea4-4650-8a9d-428b7ec24171 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.887178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.890242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.890536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.890813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.891468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:31.891775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.893652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.896305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff9c1c-ac04-4b48-a97c-d4152c60eab6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:31.922526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 369/745] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:31.931750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:31.958108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:31.958723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:31.959051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:31.959328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:31.960061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:31.960401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:31.961833 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:31.990592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 377/746] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:31.995572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.006094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.126044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd96e0-c3f2-4b3d-96f7-fb4aa0a57aa4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.149745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852505-3856-485c-9017-2df000c76c6e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.160558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.171031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.171884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.172399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.172862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.182599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.184948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.188007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.188656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.188937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.189213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:32.198761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:32.199639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:32.201121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:32.202285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 370/747] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:32.203063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.272596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.274217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.313545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.320061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d80d18-e09b-4800-92f6-6f0fa7bcabfb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.358501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.359222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.359583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.359923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.360700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.362226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.364025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:32.372143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:32.373967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:32.374806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 371/748] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:32.375597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.391121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 378/749] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:03:32.391916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.487353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486bd2b-f1b9-47a9-9eef-ddc2c33ec25c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.502149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df82706-d2b9-43a7-88d6-344959401dc3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.527003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.527858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.528353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.528819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.529792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:32.530198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.531951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.539719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.540373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.540655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.540931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.541560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:32.542015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.543380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.548562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 379/750] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:32.549428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.550347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 372/751] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:32.551254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.576594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.599905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.659749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dfcd8-472f-4923-89ba-cd1e70e0a3a5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.662130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f42574-d94c-446e-b868-5abc74577e5d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.701333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.701958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.702234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.702502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.702662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.703591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.704027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.704478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.705360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=37ca19cdbc204ba995b2d8d46a914fc3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:32.705854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.707419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.713311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 373/752] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/role_assignments?user.id=37ca19cdbc204ba995b2d8d46a914fc3&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:32.714262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.722345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.728843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:32.749678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:32.750713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:32.751615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:32.751615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:32.751615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:32.756524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 380/753] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:32.763744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.861004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.879406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420fb5f-8e8c-4c73-96df-f2f9972beab1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:32.917264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c105599-77d8-46cd-9217-20328623253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:32.917798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c105599-77d8-46cd-9217-20328623253b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:32.918072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c105599-77d8-46cd-9217-20328623253b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:32.918346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c105599-77d8-46cd-9217-20328623253b None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:32.918976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c105599-77d8-46cd-9217-20328623253b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:32.919833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c105599-77d8-46cd-9217-20328623253b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:32.924229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c105599-77d8-46cd-9217-20328623253b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:32.928101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 374/754] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:32.933664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c105599-77d8-46cd-9217-20328623253b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:32.966580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:32.974834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fa37a-fb51-4bcd-9c77-89d21d655b10 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:33.040400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:33.041020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.041330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.041612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.042268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:33.042571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:33.043936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:33.071140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 381/755] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:33.078066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:33.103210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c105599-77d8-46cd-9217-20328623253b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:33.146762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:33.147376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.147660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.147933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.148667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] RBAC: Adding query filter params (interface=public, service_id=377c66e7b9be45d38769e86f02a48f2c, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:33.148977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:33.150356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:33.171456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 375/756] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:03:32 2022] GET /identity/v3/endpoints?service_id=377c66e7b9be45d38769e86f02a48f2c&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:33.177050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:33.209424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:33.273632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520f18d-a739-4da4-945b-502dbdae118a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:33.313493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:33.314146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.314428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.315136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.322724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:33.324181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:33.339053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:33.339998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:33.340956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 382/757] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:33.353460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:33.387552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed251a-4e7e-4e70-a6c2-625908fe82a7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:33.388717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:33.434027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d39c4-736e-4cbf-8c04-852e8163178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:33.435500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d39c4-736e-4cbf-8c04-852e8163178c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.435982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d39c4-736e-4cbf-8c04-852e8163178c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.436482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d39c4-736e-4cbf-8c04-852e8163178c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.437298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d39c4-736e-4cbf-8c04-852e8163178c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:33.438909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d39c4-736e-4cbf-8c04-852e8163178c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:33.444653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 376/758] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:03:33 2022] GET /identity/v3/services => generated 772 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:33.452224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb4b9-e68f-449c-8c12-365169a9fd6e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.452994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb4b9-e68f-449c-8c12-365169a9fd6e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.453538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb4b9-e68f-449c-8c12-365169a9fd6e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.455495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 377/759] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:33.463041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d8ebf-4e92-4e07-97b2-0de95058c544 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.463659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d8ebf-4e92-4e07-97b2-0de95058c544 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.464135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d8ebf-4e92-4e07-97b2-0de95058c544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.498642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-571d8ebf-4e92-4e07-97b2-0de95058c544 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:33.500245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-571d8ebf-4e92-4e07-97b2-0de95058c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:33.509157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e1ab5-c912-468c-b385-87ed75344190 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:33.576368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:33.577103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.577385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.577658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.578375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:33.578680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:33.581062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:33.585722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 378/760] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:33.587308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1f37e-df5a-4cdd-96e7-4101451f8df5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.587788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1f37e-df5a-4cdd-96e7-4101451f8df5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.588308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1f37e-df5a-4cdd-96e7-4101451f8df5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.590116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 379/761] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:33.600253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 383/762] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:33.601884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6692324-2613-417c-b499-364342f9f763 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.602238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6692324-2613-417c-b499-364342f9f763 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.602579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6692324-2613-417c-b499-364342f9f763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.611886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:33.617677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:33.625451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c6692324-2613-417c-b499-364342f9f763 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:33.627116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c6692324-2613-417c-b499-364342f9f763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:33.735527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 380/763] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:33.738640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:33.771309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34094-63ca-4054-ac58-6fe59cebfcf1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:33.809266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:33.810462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:33.810745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:33.811013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:33.811745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=25f03b43ee624965b53e9e3163fbeb13) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:33.812050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:33.813508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:33.820000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 384/764] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:33 2022] GET /identity/v3/role_assignments?user.id=25f03b43ee624965b53e9e3163fbeb13&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:33.874190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6692324-2613-417c-b499-364342f9f763 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.014542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6692324-2613-417c-b499-364342f9f763 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.065228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.086935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.087579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.087857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.088130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.088755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.090133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:34.100452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:34.113198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:34.114687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:34.115955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:34.118109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 381/765] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:33 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:34.120327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.209290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de7371-2e14-475f-b3b7-367835f4f120 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.254627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f2166-a306-418c-b991-72aeaab836de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.255273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f2166-a306-418c-b991-72aeaab836de None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.255556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f2166-a306-418c-b991-72aeaab836de None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.255826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f2166-a306-418c-b991-72aeaab836de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.263496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:34.268764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f2166-a306-418c-b991-72aeaab836de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-176f2166-a306-418c-b991-72aeaab836de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:34.274641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:34.275784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:34.276670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:34.283689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 385/766] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:34.285226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f2166-a306-418c-b991-72aeaab836de None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.301135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ca03e-fba9-4413-871d-871b49547ed8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.393281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f2166-a306-418c-b991-72aeaab836de None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.397024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.397533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.397818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.398100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.398741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] RBAC: Adding query filter params (name=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:34.399048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.400536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:34.405920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 382/767] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:34.411381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.433125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.434065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.434530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.435607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.436610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:34.437098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.438679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:34.452370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 386/768] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:34.461426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.525341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:34.557931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:34.608012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c5f-726b-459d-8f75-9d3b2d9d5600 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.622745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07c0e6-fdd1-46d5-8f19-119be4181e95 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.649242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.649868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.650146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.650423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.673600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.675086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:34.692947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.693556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.693834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.694105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.694733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:34.695035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.696519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:34.703679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:34.704698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:34.705668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:34.705668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:34.705668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:34.707906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 387/769] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:34.708542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 383/770] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:34.713499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.714720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.889974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b35a4-6bcb-4cf7-a2cc-0bc65ca230ba None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.907553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:34.918132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea89e6-497f-4f83-86a2-6859126a329a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:34.940783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.941401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.941681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.941950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.942602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:34.942905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.944298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:34.961156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:34.965188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 388/771] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:34.973564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:34.988922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:34.989584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:34.989873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:34.990145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:34.990958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] RBAC: Adding query filter params (interface=public, service_id=24ffec3b5463407aa8c0811424b55469, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:34.991298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:34.992711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:35.002766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 384/772] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/endpoints?service_id=24ffec3b5463407aa8c0811424b55469&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:35.011714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:35.114778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7495955-f40a-47a5-a09f-e75f688a0b42 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:35.129889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9d625-390b-4e10-980d-a5a77059facc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:35.162984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:35.164102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.164732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.165183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.168346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8082ac6-fa5f-49e1-bac6-86cac9e9718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:35.168952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082ac6-fa5f-49e1-bac6-86cac9e9718c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.169271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082ac6-fa5f-49e1-bac6-86cac9e9718c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.169757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082ac6-fa5f-49e1-bac6-86cac9e9718c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.170479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8082ac6-fa5f-49e1-bac6-86cac9e9718c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:35.171857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8082ac6-fa5f-49e1-bac6-86cac9e9718c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:35.173291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:35.175488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:35.177583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 385/773] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:35.178877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf820f-35dc-4f28-9a6a-2e770b89dfab None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.179167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf820f-35dc-4f28-9a6a-2e770b89dfab None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.179447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf820f-35dc-4f28-9a6a-2e770b89dfab None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.185087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 386/774] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:35.187009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.187296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.187575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:35.206383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:35.208541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:35.210756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 389/775] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:03:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:35.217347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:35.229737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:35.231126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:35.281936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:35.338180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:35.366934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 387/776] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:35.395678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31206-8e06-4f83-a175-89b74ab540fb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:35.432046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:35.432606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.432885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.433160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.433892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:35.434196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:35.435952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:35.454301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 390/777] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:35.459603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:35.577645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ba8ba-71b1-44f1-9ec1-cf56a068ecbd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:35.616428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:35.617407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.617869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.618332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.629584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:35.631293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:35.646993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:35.648000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:35.648962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:35.648962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:35.648962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:35.650794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 388/778] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:35.656848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:35.697404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:35.764440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:35.789159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:35.793658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7032c-38df-4f7b-a928-518cbc2c4a0c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:35.859651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:35.860286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.860567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.860840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.861487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:35.861790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:35.863154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:35.869915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bec30-d6ea-44cc-a4c5-3b5215b61186 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:35.892183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 391/779] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:35.901205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:35.921561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:35.922300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:35.922834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:35.923428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:35.936604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:35.941959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:35.942910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:35.943793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:35.948004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 389/780] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:35.949167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.064607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e186bef-8d95-4e7a-b2a9-9c6d2067f0a4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.077258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:36.082846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f671dee-bfe9-44e6-a69e-906c47836b4c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.105121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.105788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.106071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.106346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.114159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.115630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.124059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.124950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.125434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.125935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.126793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:36.127297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ef3be9a2-a28a-40d5-8656-3d01daaa463a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:36.128015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:36.128915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:36.129804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:36.131622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd18604-c8c6-45a4-90e8-2358545eb57c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.133897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 392/781] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:36.135256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e977ff-95b2-494a-982e-a3a3263839b7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.135550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e977ff-95b2-494a-982e-a3a3263839b7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.135826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e977ff-95b2-494a-982e-a3a3263839b7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.141868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 393/782] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:36.141868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e977ff-95b2-494a-982e-a3a3263839b7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.147036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 390/783] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:36.148500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb1a7-615f-4f2f-999e-c3562072654c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.148934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb1a7-615f-4f2f-999e-c3562072654c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.149355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb1a7-615f-4f2f-999e-c3562072654c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.150948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 391/784] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:36.152149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.152726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.153178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.153285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:36.172978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:36.174181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:36.247539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 392/785] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:36.248134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.263766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e977ff-95b2-494a-982e-a3a3263839b7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.314003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.314642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.315045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.315327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.316377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:36.316692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.318387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a559a-0625-4a5a-8a23-4bbd8f51b9e3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.335156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 394/786] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:36.336530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.336849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.337531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.353952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:36.360629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:36.361843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:36.370630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfad21-0f47-49be-b3c1-7d584b142156 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.421008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.422253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.423158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.423920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.429040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:36.440174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.440662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 395/787] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:36.441528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.442425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:36.460938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:36.461902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:36.462801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:36.462801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:36.462801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:36.465421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 393/788] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:36.477542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.594566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be2700-7e5c-48a0-9655-966eeefeb667 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.624852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227863c2-583f-4437-963d-ec46d0c8efec None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.631779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.632498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.632812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.633059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/1f43ff51c5f84d34978c46bce6db67ee/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.635211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=1f43ff51c5f84d34978c46bce6db67ee, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.636950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.656662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 396/789] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:36 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/1f43ff51c5f84d34978c46bce6db67ee/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:36.657652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.689248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.689878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.690159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.691456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.692143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:36.694567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.696067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.728252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 394/790] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:36.734613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.741219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:36.772847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:36.818251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f3f82-4b4f-4477-a1dd-fc461f345e94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.852285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47089661-79fe-46ad-a7c1-058820e666d0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:36.860972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.861605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.861887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.862163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.862752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.865855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:36.878168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:36.879224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:36.880073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 397/791] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:36.880963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:36.897582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:36.898224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:36.898505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:36.898780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:36.906625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:36.908241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:36.921156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:36.922349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:36.923533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:36.929553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 395/792] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:36.930495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.020153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce73da-d286-4915-aa4f-0fef054f26a0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.048157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.061868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.062684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.063069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.063445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.076880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:37.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:37.083521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:37.084675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 398/793] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:37.089849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.140706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.175346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0dba5-4326-41b5-a915-ab60e44faa6e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.210093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328478b-8827-42ab-a494-887ab5e939f6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.214387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.214993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.215419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.215744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.216494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:37.216801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.218310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.223564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 396/794] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Aug 18 11:03:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:37.224584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.252105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.252922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.253543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.253824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.254860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:37.255170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.256950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.276442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 399/795] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:37.277347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.333175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.351011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182d60f-72c8-4de6-b374-a0fc51bc6a1f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.382137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.393966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.394623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.394972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.395334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.396078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:37.396818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.398097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.411257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f257b-61bb-4082-8aaa-690ac40fc868 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.413291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 397/796] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:37.414727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.465858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.468047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.468829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.469328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.470563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:37.471087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.473258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.482971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 400/797] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:37.484395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.525377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a14de-65a9-4084-b3a3-74e28b7c81f6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.562595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.563211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.563488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.563754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.574353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.576773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.590450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54175578-1079-4530-a3bf-a14127826a3f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-89288a4b-2601-4353-8723-3d9bc02dc6a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:37.592689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:37.593608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:37.594478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:37.594478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:37.594478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:37.594478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 398/798] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:37.595119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ef346-29a8-442a-8a02-51f4f9e7d931 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.595361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ef346-29a8-442a-8a02-51f4f9e7d931 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.595634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ef346-29a8-442a-8a02-51f4f9e7d931 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.600939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 399/799] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:37.600939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ef346-29a8-442a-8a02-51f4f9e7d931 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.603439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.633379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.634014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.634309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.634571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.646328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.646778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.648143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:37.662633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:37.663591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:37.664523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:37.664523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:37.664523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:37.664523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 401/800] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:37.664523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.713941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ef346-29a8-442a-8a02-51f4f9e7d931 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.751307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11448748-ac74-49b1-8bf2-c8ba99feabf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.751917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11448748-ac74-49b1-8bf2-c8ba99feabf7 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.752222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11448748-ac74-49b1-8bf2-c8ba99feabf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.752490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11448748-ac74-49b1-8bf2-c8ba99feabf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.753151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11448748-ac74-49b1-8bf2-c8ba99feabf7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.754517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11448748-ac74-49b1-8bf2-c8ba99feabf7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.777174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10a8a-32d6-42c1-a5e1-8626d8cc97d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:37.784396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 400/801] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:03:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:37.787149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.787768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.788393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.817907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:37.820632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:37.823964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:37.824612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:37.824883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:37.825145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:37.827651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:37.827651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:37.828661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:37.843720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 402/802] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:37.844768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.862746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.933184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 401/803] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:37.934710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:37.969142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:37.976825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4ef69-9886-43e3-b7b7-5cfc7edff4e0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.022805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.023769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.024310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.024857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.025817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:38.026517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.028293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.044114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 403/804] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:38.050059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.059326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caaa56-50d4-4a5c-88ab-439f262b1e93 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.108857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.109804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.110095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.110366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.119099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.121341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:38.133954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:38.134881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:38.135782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:38.136772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 402/805] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:38.139401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:38.145348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.173360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731bd04-e9d4-48d0-b686-fa08646c195e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.210385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.211045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.211330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.211632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.220798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.222248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:38.234404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:38.236651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:38.238897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 404/806] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:38.240589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:38.244281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.257449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d95c5-e910-41a8-8809-39a2763f0b6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.295394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.296007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.296315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.296590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.297284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:38.297589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.299025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.314110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 403/807] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:38.315026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.385665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622c703-7d0a-4285-bfbf-56a522261f77 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.424561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.425492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b273-84bd-48e2-82f9-f83b67aaeb42 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.425733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.426226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.426688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.427659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:38.428219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.429827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.431838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:38.450161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 405/808] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:38.451528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.463818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.464408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.464687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.465595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.467770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:38.477524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:38.478515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:38.479349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 404/809] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:38.480125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.501156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:38.577518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b94b-f725-45c0-9770-54dbf1509ab6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.609552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfbc91-e092-439e-9e63-07792d8aa74c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.617093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.618115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.618614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.619098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/37ca19cdbc204ba995b2d8d46a914fc3/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.621452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=37ca19cdbc204ba995b2d8d46a914fc3, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.623389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.640610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 406/810] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:38 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/37ca19cdbc204ba995b2d8d46a914fc3/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:38.641570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.656712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.659441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.659441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.659699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.691395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.692920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:38.722826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:38.723826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:38.724753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:38.724753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:38.724753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:38.726540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 405/811] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:38.729888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.743527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:38.816445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a83dd-8ef6-4337-a3cf-cf38a6bfcd14 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.821628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:38.869141 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae37461-efcf-4481-b400-a0c130472726 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:38.908833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.909442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.909716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.909983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.910634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:38.910933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.912369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd5ea6-d575-4530-8072-3022f451d466 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.926453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 406/812] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:38.927749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87972-f60f-4e5b-b55b-6453bf5b2d14 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.928029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87972-f60f-4e5b-b55b-6453bf5b2d14 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.928325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87972-f60f-4e5b-b55b-6453bf5b2d14 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.929766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 407/813] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:38.936303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc87972-f60f-4e5b-b55b-6453bf5b2d14 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:38.980815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:38.981368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.981677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.981948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:38.982571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] RBAC: Adding query filter params (name=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:38.988227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:38.988227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6978797-9128-4bc9-b0f5-5ee688e3f75d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:38.989656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 407/814] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:38.990928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:38.991219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:38.991504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.020297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:39.021500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:39.082008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc87972-f60f-4e5b-b55b-6453bf5b2d14 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:39.107074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:39.124214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 408/815] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:39.124214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:39.129788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:39.131692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:39.132244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:39.134086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.145681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:39.147135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:39.165423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:39.166366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:39.167278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 408/816] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:39.176078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:39.220580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:39.305595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc02d00-fc29-4832-b838-6e78ef90c220 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:39.353739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63c9f9-a364-49c8-a122-5068e2c1a3dd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:39.370726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:39.371407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:39.372791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:39.373503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.375553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] RBAC: Adding query filter params (type=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:39.375921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:39.378139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:39.389751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 409/817] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:39.396376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:39.419759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:39.419759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:39.420033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:39.420396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.421238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:39.421633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:39.423200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:39.440303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 409/818] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:39.442394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:39.495460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:39.540260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cebd10-0693-4103-868c-9d3f6384c118 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:39.555319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:39.558641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa32818-218e-4f61-80d6-dfc2c01e7909 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:39.581216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:39.581976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:39.582350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:39.582716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.583445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:39.584979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:39.597613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:39.598520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:39.599042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:39.599482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/25f03b43ee624965b53e9e3163fbeb13/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.615422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=25f03b43ee624965b53e9e3163fbeb13, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:39.624455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:39.632426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 410/819] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:03:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:39.634442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:39.658931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 410/820] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:39 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/25f03b43ee624965b53e9e3163fbeb13/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:39.798812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ad79f-0a95-410a-9c6b-5848ca1071a7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:39.842246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:39.843317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:39.843799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:39.844286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:39.857922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:39.863192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:39.864152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:39.865027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:39.867781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 411/821] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:39.877379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:39.991657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:40.001006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d2d6a-845f-4f6c-8a31-5b2b14a0c61a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.047544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.048149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.048468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.048740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.049402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:40.049701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:40.051238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:40.060811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 411/822] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:40.065934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:40.092735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:40.174647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019f41-9aa8-437e-9e3e-c0b80645d959 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.209723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.210367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.210646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.210915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.223856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:40.225369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-61b5bd84-5b55-4817-8752-6816f5314593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:40.240141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:40.241148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:40.242026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:40.242026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:40.242026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:40.244069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 412/823] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:40.249928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:40.287605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608e69c-320a-4e88-af21-ce133f5746b5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.288273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608e69c-320a-4e88-af21-ce133f5746b5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.288743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608e69c-320a-4e88-af21-ce133f5746b5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.290516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 413/824] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:40.296052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.296640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.297105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.315941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:40.317493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:40.361976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351994c8-446c-4f43-9a15-73acab9c2a6a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.392456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 414/825] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:40.397567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.398175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.398453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.398726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.399397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:40.399698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:40.401102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:40.414612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 412/826] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:40.419814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:40.432520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:40.525672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53976670-993e-4adc-b7bd-5aceed5c1ae1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.560753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.561378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.561651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.561917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.569159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:40.570587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:40.582649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:40.584051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:40.585067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 415/827] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:40.589961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:40.608558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:40.676620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:40.699795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12860d-9e58-4033-b0f2-ad7e18567290 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.737942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.738551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.738916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.739152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.739815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:40.740117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:40.741562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77986b04-63b1-409c-b74b-c2b03a2063ed None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:40.741687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:40.760967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 413/828] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:40.762314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527779d-c7eb-4c85-829b-4275337d9597 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.762693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527779d-c7eb-4c85-829b-4275337d9597 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.763630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527779d-c7eb-4c85-829b-4275337d9597 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.765152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 414/829] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:40.767060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527779d-c7eb-4c85-829b-4275337d9597 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:40.787162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7e193-3fe4-4f96-a64d-f15fd3c84f32 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.824731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.826073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.826364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.826737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.827325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:40.828796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ede9e7c2-c80d-497c-ae58-2d7228fb8188 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:40.838373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:40.839393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:40.840680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 416/830] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:40.842038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.842038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.842321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.856235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:40.860776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:40.861981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:40.891304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527779d-c7eb-4c85-829b-4275337d9597 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:40.931613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:40.932138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:40.932447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:40.932720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:40.936329 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 417/831] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:40.937217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.015270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.022202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.023826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.069581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2a757-8247-47e9-8f41-efdbd23d14c5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.105808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.111369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.111369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.111369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.111369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:41.111369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.112896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.118490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.125492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 418/832] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:41.127108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.156636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 415/833] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 11:03:41.161854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.247990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1c1c-31d1-48fc-aeb7-c4eb7580b481 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.271555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.284858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.285600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.286134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.286243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.286882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:41.287170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.288552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.288726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b285c6-9126-45c7-9726-be5a1bebf748 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.301755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 419/834] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:41.301755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.335995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.337185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.337574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.338124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.351658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.353322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:41.372940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:41.374019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:41.374904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:41.374904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:41.374904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:41.375221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 416/835] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:41.380400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.412305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.425887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25760d-2f9b-4f3d-a3a1-f720f5aa2bdf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.465041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.466405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.466542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.466959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.478646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:41.484862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:41.485830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:41.486684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 420/836] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:41.493691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.505744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b7b31-b931-491d-91bd-65549d45cb8b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.543387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.544028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.545018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.545320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.546079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:41.546378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.547742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.561435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.561877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 417/837] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:41.568441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.639496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27bf1-00a6-44cd-bb96-2d10235a9d36 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.660730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.679656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550ec2c-0a2f-40b9-b550-1f01a3bfa646 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.687623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.688977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.689410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.689831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.690668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:41.691488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.693515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.710195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 421/838] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:41.710975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.726418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.727052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.727329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.727607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.735319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.736878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:41.749655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:41.750587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:41.751513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:41.755872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 418/839] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:41.756719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.836687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da257947-1fb6-4e5c-8e32-ed5b4627ea2e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.846823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.863380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2faafd-417a-45eb-bed2-35d07dc7baac None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:41.877984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.878608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.878880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.879152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.890546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.892002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.900010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:41.900661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.900943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.901216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.901908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:41.902212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:41.903707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c50da8-ddb2-45c3-ad9c-22e2fc0b3753 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d0fa3e5e-4118-4389-94d6-fd075d94dfee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:41.906991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:41.907936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:41.908923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:41.908923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:41.908923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:41.908923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 422/840] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:41.909638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700a27-09cc-4f19-8e2d-0cf9b3c583ab None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.909916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700a27-09cc-4f19-8e2d-0cf9b3c583ab None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.910190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700a27-09cc-4f19-8e2d-0cf9b3c583ab None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.911551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 423/841] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:41.916050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e700a27-09cc-4f19-8e2d-0cf9b3c583ab None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:41.928329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 419/842] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:41.928329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:41.928329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:41.928329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:41.933798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:41.946932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:41.948162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:42.023974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 420/843] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:42.025062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.039065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e700a27-09cc-4f19-8e2d-0cf9b3c583ab None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.078638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.079560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.080061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.080598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.081438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:42.081940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.083506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.097476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 424/844] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:42.103034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.127496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:42.163207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2226a86-f401-4926-8304-65c1ff35557a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.215444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.216593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.216739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1165d2-ab90-46a8-9833-443ac8b15504 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.217216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.217670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.218847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=1f43ff51c5f84d34978c46bce6db67ee) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:42.220224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.221963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.223978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:42.236816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 421/845] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/role_assignments?user.id=1f43ff51c5f84d34978c46bce6db67ee&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:42.261194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.262028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.262374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.262715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.271985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.273659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:42.287253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:42.288256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:42.289209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 425/846] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:42.295072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.318472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.423072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:42.434842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef500e-e3bb-4608-acaa-27bb44dd4f95 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.450128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8e300-78d6-4bcf-8b1c-c6e1e4bbb332 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.475649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.476290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.476572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.476845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.477572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:42.477876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.479308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.489919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.490550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.490828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.491097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.500085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 422/847] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:42.505211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:42.505634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:42.510403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:42.511381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:42.512307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 426/848] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:42.521147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.576262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:42.625336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b323233-2118-4dd2-bf9e-da8f60d5a3ed None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.656352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b5731-4734-4c3b-bdc2-fe331287cee2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.661905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.662588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.662940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.663283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.702951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.703609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.703891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.704158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.704864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:42.705166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.707143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.724530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 427/849] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:42.725438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.730164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:42.743352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.744958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.803513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:42.858928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 423/850] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:03:42.859927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.867180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853955d-680f-4779-9f77-285519b1d2bf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:42.903116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:42.904867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:42.905370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:42.905856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:42.917491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:42.918943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:42.936846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:42.937862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:42.938740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:42.938740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:42.938740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:42.943549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 428/851] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:42.945456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:42.973015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5675f5-596b-48c0-bcc0-bbc068809455 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.012438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.013556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.014069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.014548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.016869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.027218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.028081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:43.044961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:43.046149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:43.047248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:43.047248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:43.047248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:43.052685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 424/852] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:43.053761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.065797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.081637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e434a-121e-426d-8dcf-fbccee6a45a4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.118510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.119429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.119908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.120428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.121262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:43.121763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.123624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.140860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 429/853] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:43.142080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.163732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cfb14-a729-4798-8f5e-c32ba8e44e6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.204864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.205562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.206015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.206140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.206779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:43.207087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.208501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.226163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 425/854] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:43.227493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.257481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f75a4-1a47-4276-8f2c-0f5eeef6d677 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.274390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.299942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.300749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.300956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.301223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.302487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.309536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.311032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d2cb75ef-a36c-4987-9f7b-1800bf658c0e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:43.322808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:43.323748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:43.325827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 430/855] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:43.330406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6840211-07ce-41e6-a2e9-0ee98cf3f73f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.330520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6840211-07ce-41e6-a2e9-0ee98cf3f73f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.330740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6840211-07ce-41e6-a2e9-0ee98cf3f73f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.336142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 431/856] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:43.336142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6840211-07ce-41e6-a2e9-0ee98cf3f73f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.344800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59662da-82d3-48e0-8560-3f37a02d6e33 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.381238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.381797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.382204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.382480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.390360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.391797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-98926b16-15c1-4658-a7ef-9e64aa1ede52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:43.402743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:43.404320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:43.405393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 426/857] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:43.406326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.406326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.406480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.424627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:43.425824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:43.456070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6840211-07ce-41e6-a2e9-0ee98cf3f73f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.491367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860329f-7135-422f-836c-1868566ce289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.491993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860329f-7135-422f-836c-1868566ce289 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.492294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860329f-7135-422f-836c-1868566ce289 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.492528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860329f-7135-422f-836c-1868566ce289 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.493241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860329f-7135-422f-836c-1868566ce289 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:43.493560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860329f-7135-422f-836c-1868566ce289 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.494952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860329f-7135-422f-836c-1868566ce289 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.504527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 427/858] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:43.504527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.517703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 432/859] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:43.524096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860329f-7135-422f-836c-1868566ce289 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.568650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.585509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.634383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda4ab2-cc8e-46b5-82e6-4ac32afaa816 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.638443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860329f-7135-422f-836c-1868566ce289 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.670738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.671418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.671761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.672107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.672931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:43.673302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.674981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.676134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.677366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.677878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.678314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.688052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 428/860] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:43.694318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.753739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.755648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.789295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.806106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e4cd7-5553-41c7-8c6f-ce2f04ca005e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:43.825973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:43.843756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:43.844178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:43.844493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:43.844764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:43.845487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=37ca19cdbc204ba995b2d8d46a914fc3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:43.845794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:43.847216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:43.852961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 429/861] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/role_assignments?user.id=37ca19cdbc204ba995b2d8d46a914fc3&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:43.853737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.877233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 433/862] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 11:03:43.878304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:43.980494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59e594-96b1-4652-8999-2ff7f4cb1223 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.010542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5db4d-eb65-47bf-9393-80e2e3c6f04f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.020841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.021459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.021761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.022012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.022656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.024066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.045997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 430/863] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:03:43 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:44.052697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:44.067866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.068690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.068971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.069236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.085082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.086349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:44.104416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:44.105456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:44.106371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:44.106371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:44.106371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:44.108452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 434/864] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:44.114782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:44.167759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:44.234370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06bc1-e1d5-46cb-afde-a0e3a55ef7fc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.276002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.276637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.276914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.277180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.277820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:44.278116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.279900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154bd51-9355-4ddf-b3f9-928b0e2ce2f5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.296403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 431/865] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:44.300182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:44.374936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f388214-d50f-4f74-8deb-a25c529a60b6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.375460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f388214-d50f-4f74-8deb-a25c529a60b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.375537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f388214-d50f-4f74-8deb-a25c529a60b6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.377553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 432/866] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:44.382622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.383021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.383305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.406119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:44.408064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:44.413413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8236b15-d3d4-4a07-8891-33fcf3e1358f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.451963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.452641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.452914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.453188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.460820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.462274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:44.475543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:44.476518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:44.477410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 435/867] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:44.483013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:44.496928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 433/868] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:44.532906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:44.593826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fc0f8-bf5c-4478-943d-3b177656e785 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.629075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.629673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.629945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.630217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.630963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:44.631259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.632774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.650065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 436/869] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:44.651459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:44.684274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:44.761642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f27d74-969b-4899-aac9-bf07da67fec5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.764311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:44.801438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.802044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.802323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.802696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.803544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=25f03b43ee624965b53e9e3163fbeb13) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:44.803851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.805397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.811984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 434/870] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:44 2022] GET /identity/v3/role_assignments?user.id=25f03b43ee624965b53e9e3163fbeb13&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:44.875693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:44.883281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f842a-ca17-42bd-bc38-a5e048fa7b80 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:44.923793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:44.924867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:44.925363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:44.925850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:44.926640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:44.928236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:44.938453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:44.939474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:44.940456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:44.941803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 437/871] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Aug 18 11:03:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:44.948226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:45.064977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:45.072028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26bf36-c1a3-4f0c-9f24-66688647982e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:45.110794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:45.111407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:45.111783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:45.112065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:45.112735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:45.113079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:45.114413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23be40-8f8b-4966-9c65-86e3e6694e11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:45.120090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 435/872] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:45.126843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:45.236474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de951ebf-1239-408a-bf85-6717435c2377 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:45.273107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:45.273707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:45.273980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:45.274381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:45.275034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:45.275330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:45.276726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848702f-05ec-4f80-89f9-15097e8fb367 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:45.282070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 438/873] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:46.006873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047f797-624f-4d25-a070-d0a856683fa8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.007437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047f797-624f-4d25-a070-d0a856683fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.007684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047f797-624f-4d25-a070-d0a856683fa8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.009430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 436/874] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:46.015370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed3ce-3e1f-4920-bb03-85edc9558cf1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.015793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed3ce-3e1f-4920-bb03-85edc9558cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.016087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed3ce-3e1f-4920-bb03-85edc9558cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.034646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ad6ed3ce-3e1f-4920-bb03-85edc9558cf1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:46.035830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ad6ed3ce-3e1f-4920-bb03-85edc9558cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:46.061364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:46.108559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34d9c2-51da-4d5f-ab45-e0053e6ac1cb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.109186 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34d9c2-51da-4d5f-ab45-e0053e6ac1cb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.109691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34d9c2-51da-4d5f-ab45-e0053e6ac1cb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.111443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 437/876] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:46.111571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 439/876] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:46.116811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.117092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.117361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.135235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:46.136438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:46.205131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 440/877] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:46.392588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34d9c2-51da-4d5f-ab45-e0053e6ac1cb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:46.478166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:46.499890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34d9c2-51da-4d5f-ab45-e0053e6ac1cb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:46.545965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:46.546875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.547158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.547427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.548060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:46.550149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:46.561459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:46.562486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:46.563334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:46.568065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 438/878] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:46.569340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:46.589125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:46.606356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1870fa-59b9-4926-a26a-919a9f890d11 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:46.650110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:46.650657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.650930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.651196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.651777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:46.653167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:46.665212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:46.666260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:46.667123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 441/879] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:46.673294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:46.673780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:46.689335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebd8a-a40a-47a8-b8f3-61829a765cb1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:46.731980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:46.732743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.733043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.733427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.734057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:46.734354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:46.735876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:46.744259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 439/880] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:46.749501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:46.805494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c4475-f32a-4dfb-b313-9d0fa0b56c95 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:46.847991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:46.849651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:46.850278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.850552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.850818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.851601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:46.851966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:46.853634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:46.865491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 442/881] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:46.867000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:46.869774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d664bda-090c-4d4b-9c68-c68c7979476c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:46.890418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:46.913961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:46.914568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.914850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.915115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.915744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:46.916043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:46.917539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321ba11-72cb-49d7-8546-216cd130ec06 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:46.928334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 440/882] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:46.928334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e5bb8-55cf-4893-b9c3-6fe98bf13e19 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.928334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e5bb8-55cf-4893-b9c3-6fe98bf13e19 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.928334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e5bb8-55cf-4893-b9c3-6fe98bf13e19 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.928334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 441/883] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:46.931794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:46.932076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:46.932378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:46.957653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:46.959700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:47.011935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a6c63-6969-47f0-b784-578e58b92a3a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.055955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.056683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.056894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.057153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.057791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:47.058269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:47.059888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:47.070245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 443/884] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:47.070437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 442/885] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:47.072491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:47.156226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:47.176677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e29ae6-8f71-4c3a-917e-ebfa6b43b71b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.210836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.211437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.211714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.211985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.212710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] RBAC: Adding query filter params (interface=public, service_id=0aaa07325866442284e78a3b70c004a0, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:47.213013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:47.214371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd333cec-5784-453d-abc8-850d639fcb6c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:47.222959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 444/886] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:03:47 2022] GET /identity/v3/endpoints?service_id=0aaa07325866442284e78a3b70c004a0&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:47.228366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:47.247688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07e41-04dc-4561-a56b-9108b1ad8313 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.247974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07e41-04dc-4561-a56b-9108b1ad8313 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.248282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07e41-04dc-4561-a56b-9108b1ad8313 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.249787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 445/887] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:47.255456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.255756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.256041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.274489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:47.275681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:47.334071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:47.352459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 446/888] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:47.356800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6b687-5cf2-4b23-9bae-35d4e0839494 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.377457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:47.396747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.397843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.398263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.398778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.399522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:47.401587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:47.410101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 443/889] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:03:47 2022] GET /identity/v3/services => generated 1011 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:03:47.470494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:47.501903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903d64e-01ac-423c-a767-178d20a46b98 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.540616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.541534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.541953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.542359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.553955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:47.558849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:47.560446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:03:47.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:47.563541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 447/890] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Aug 18 11:03:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:47.569329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:47.629031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:47.690491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3d3e0-b446-4fe0-93bf-535e64f56091 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.728553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.729077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.729356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.729631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.730359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:47.730662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:47.732649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:47.741584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 444/891] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:03:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:47.746894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:47.763096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:47.766544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36338767-9559-434c-8ed2-8753d57cb0bd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.816179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.817053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.817501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.817911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.818696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:47.821059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:47.826426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:47.844876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 448/892] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:03:47 2022] POST /identity/v3/services => generated 269 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:03:47.884688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1429-0070-4f09-a279-b76fb97e1199 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:47.936037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:47.936802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:47.937092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:47.937362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:47.949547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:47.951939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:47.968919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:47.969925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:47.971255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:47.971255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:47.971255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:47.971255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 445/893] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:47.976066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:48.105898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a133f95-fdbb-4be5-a2d4-791391b6a17a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:48.145120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:48.146052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:48.146563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:48.147007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:48.147816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:48.148321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:48.149868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:48.163657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 449/894] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:48.169415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:48.229610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:48.277375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec28f3c-51b3-4ebd-9866-312d4bed1ef3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:48.314360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:48.315405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:48.315860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:48.316359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:48.324885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:48.327112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:48.339045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:48.340015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:48.342025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:48.344341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 446/895] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:03:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:48.353343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:48.424304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:48.472486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df861b9c-3610-4074-8759-145621153ac2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:48.508867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:48.509489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:48.509766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:48.510087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:48.510813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:48.511112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:48.512556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:48.531316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 450/896] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Aug 18 11:03:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:48.536891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:48.645994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa74767-3eb6-4a96-9913-4f47c76a0289 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:48.662587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:48.681729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eece58-b82d-465a-b1eb-f0a4babf020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:48.682332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eece58-b82d-465a-b1eb-f0a4babf020e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:48.682607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eece58-b82d-465a-b1eb-f0a4babf020e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:48.682878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eece58-b82d-465a-b1eb-f0a4babf020e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:48.755345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eece58-b82d-465a-b1eb-f0a4babf020e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:48.756862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eece58-b82d-465a-b1eb-f0a4babf020e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:48.863805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 447/897] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:03:48.873986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:48.924483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a7daf-06a4-44e9-a790-3d711c50eb00 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:48.924805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a7daf-06a4-44e9-a790-3d711c50eb00 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:48.925070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a7daf-06a4-44e9-a790-3d711c50eb00 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:48.926671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 448/898] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:48.932774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:48.933057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:48.933327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:48.952540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:48.953730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:48.965437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:48.986127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d2059-a21a-4d39-8f56-16405f9294c8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:49.001361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:49.028748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:49.029802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.030290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.030813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.044680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 449/899] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:49.050732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:49.052449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:49.067954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:49.068963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:49.069902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:49.069902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:49.069902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:49.071895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 451/900] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:49.077690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:49.186264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cea7b7-0cdc-4219-a5f5-63fcc2a8c253 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:49.226952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:49.227553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.227833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.228106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.228776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:49.229076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:49.230450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:49.242355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 450/901] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:49.248312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:49.344623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:49.354346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:49.364427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4655dc-2f32-4496-8bca-87e39733c198 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:49.403312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:49.405102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.405618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.406106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.413879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:49.415796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:49.430076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:49.431694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:49.432630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:49.435222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 452/902] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:49.441164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:49.451021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64634df-6c5a-4e6e-aaa4-93e1511007b1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:49.473779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:49.488208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16822b-dc15-4d8d-859f-e738267fb422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:49.488827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16822b-dc15-4d8d-859f-e738267fb422 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.489106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16822b-dc15-4d8d-859f-e738267fb422 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.489382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16822b-dc15-4d8d-859f-e738267fb422 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.490041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16822b-dc15-4d8d-859f-e738267fb422 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:49.491402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16822b-dc15-4d8d-859f-e738267fb422 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:49.511771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 451/903] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:03:49 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:03:49.514032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b61c7-669b-456a-b71e-48246b888b12 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.514417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b61c7-669b-456a-b71e-48246b888b12 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.514694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b61c7-669b-456a-b71e-48246b888b12 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.516265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 452/904] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:49.522051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.522351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.522592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.543771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:49.545373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:49.572027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8aaa7-f138-458b-a9b8-1ac091900856 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:49.620039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:49.629525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:49.630868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.631137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.631409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.632237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:49.632559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:49.634075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637fa45-0260-44f9-a64e-2701c7845213 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:49.634980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 453/905] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:49.658668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 453/906] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:49.665136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:49.773412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345ba9-bd73-43de-b9e7-081d897b52ed None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:49.809116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:49.810636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.811108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.811564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.812506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] RBAC: Adding query filter params (role.id=6cfa4326e893449aa1e4daa24e4223e7, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=25f03b43ee624965b53e9e3163fbeb13) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:49.813007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:49.814727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c136e8-a0b4-4cf2-9e01-c4f96b76d770 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:49.820966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 454/907] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:49 2022] GET /identity/v3/role_assignments?user.id=25f03b43ee624965b53e9e3163fbeb13&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=6cfa4326e893449aa1e4daa24e4223e7 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:49.902936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0d1a2-c845-4738-8ead-b344db6373cc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.903821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0d1a2-c845-4738-8ead-b344db6373cc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.904065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0d1a2-c845-4738-8ead-b344db6373cc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.905846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 454/908] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:49.911495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:49.912067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:49.912567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:49.924333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0d1a2-c845-4738-8ead-b344db6373cc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:49.935282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:49.937115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:50.015921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:50.032769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 455/909] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:50.032919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:50.060297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0d1a2-c845-4738-8ead-b344db6373cc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.097922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.098872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.099290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.099731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.100503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.101992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:50.111454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:50.112614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 11:03:50.113433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:50.114164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 455/910] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:50.121616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:50.226049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85a16c-8613-466e-a794-1284759ec48c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.262212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.262993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.263278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.263535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.264230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:50.264537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.265894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.271025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 456/911] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:50.277566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:50.355368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:50.389824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1067f0-5455-4eeb-83b4-dba0e5eeeabe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.424590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:50.426959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.427670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.428061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.428564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.429276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:50.432449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.432449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.437209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 456/912] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Aug 18 11:03:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:50.442900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:50.454752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:50.476996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723f311-4733-4419-8a71-c647a3e1c17c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.515209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.516074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.516546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.516958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.517696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.519311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:50.529258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:50.530546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:50.531635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:50.534112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 457/913] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Aug 18 11:03:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:50.540046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:50.562749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd759a-e1f5-4b34-af8e-b9c0eaf5e157 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.600214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.600949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.601294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.601633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.602798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.604492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.631965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 457/914] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:03:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:50.670292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eda58-5592-46da-87ae-8f1ab08f7529 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.691362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:50.725966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.726586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.726914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.727189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.727827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:50.728127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.729624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.736639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 458/915] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:50.742503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:50.811696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:50.862331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee07ea-b644-416c-8363-7cbbdf375f63 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:50.903557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:50.904354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:50.904635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:50.904902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:50.905522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:50.905818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:50.907176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:50.912558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 458/916] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:50.918262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:51.033821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee6f3-44ec-44cc-86bb-24f15edbe7d4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:51.069383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:51.069998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:51.070270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:51.070538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:51.071234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] RBAC: Adding query filter params (interface=public, service_id=13b1f97eefdf4926ab7af726d0383545, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:51.071533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:51.072917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6b522-3618-4ce6-8a3b-1b18a1f41719 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:51.079420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 459/917] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:03:50 2022] GET /identity/v3/endpoints?service_id=13b1f97eefdf4926ab7af726d0383545&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:51.085067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:51.199897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf25ad6-e19a-48f4-959b-e032ab95ff46 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:51.227798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:51.237149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb472a9-17a3-4231-8dc0-d291ead385a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:51.237755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb472a9-17a3-4231-8dc0-d291ead385a1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:51.238033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb472a9-17a3-4231-8dc0-d291ead385a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:51.238305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb472a9-17a3-4231-8dc0-d291ead385a1 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:51.238908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb472a9-17a3-4231-8dc0-d291ead385a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:51.240306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb472a9-17a3-4231-8dc0-d291ead385a1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:51.246829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 459/918] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:03:51 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:03:51.589344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf5206-d756-4acc-a761-24aff366802a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:51.589863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf5206-d756-4acc-a761-24aff366802a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:51.590076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf5206-d756-4acc-a761-24aff366802a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:51.592482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 460/919] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:51.597966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1df7b-bce1-4db7-b327-57d79d026be9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:51.598419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1df7b-bce1-4db7-b327-57d79d026be9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:51.598719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1df7b-bce1-4db7-b327-57d79d026be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:51.617402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-86f1df7b-bce1-4db7-b327-57d79d026be9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:51.618608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-86f1df7b-bce1-4db7-b327-57d79d026be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:51.691787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 460/920] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:51.874546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481578c5-ac1d-47b5-9de2-996e06f9cfd9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:51.874720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481578c5-ac1d-47b5-9de2-996e06f9cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:51.874996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481578c5-ac1d-47b5-9de2-996e06f9cfd9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:51.876630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 461/921] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:51.881658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:51.881941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:51.882211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:51.899990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:51.901216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:51.968806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 461/922] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:51.986217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481578c5-ac1d-47b5-9de2-996e06f9cfd9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.042469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.110188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481578c5-ac1d-47b5-9de2-996e06f9cfd9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.146600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.146883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.147154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.147738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.149133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:52.157907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:52.158961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:52.160489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 462/923] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:52.165260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.220562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.245037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.273425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd7d1-3743-463e-b7a6-0f3c9ffe0136 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.319866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.320752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.321027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.321296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.321922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:52.322220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.323577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.332234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 462/924] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:52.337072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.368082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd1940-b6d3-44a6-aa31-049818b24d6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.376921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.418983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.420341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.420833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.421556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.436059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:52.442954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:52.443885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:03:52.444761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:52.452292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 463/925] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:52.454794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.463000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3698d-0832-498a-a5f0-ec1ddd9cf8c9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.482115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.502313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.503280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.503769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.504290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.505132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:52.505658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.507298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.518176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 463/926] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:52.519282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.585420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82220f3d-b08f-4cd7-bf40-f7b69e74f463 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.627465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f76676-abe6-4185-8c1a-147ec129d7fe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.632098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.632723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.632999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.633265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.633931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:52.634227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.635584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.652528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 464/927] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:52.653631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.675739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.676305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.676605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.677306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.678000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] RBAC: Adding query filter params (interface=public, service_id=e573cc03d9fb4244b06134fcbc17e47a, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:52.678304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.679665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.691812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 464/928] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/endpoints?service_id=e573cc03d9fb4244b06134fcbc17e47a&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:52.691978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.693427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.745626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.777222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbe7c4-9016-4d37-85f6-441d5a599173 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.809181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eff17e-d164-4a4d-96a5-772b675cc44a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:52.811854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.812521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.812802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.813809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.824536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.826554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e242190f-eaf0-4187-8c02-af13aa670b41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:52.841946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:52.843082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:52.844630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:52.844630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:52.844630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:52.844630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 465/929] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:52.844630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3ab72-9f9b-40a7-8670-c8bb247a28d7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.844630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3ab72-9f9b-40a7-8670-c8bb247a28d7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.845010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3ab72-9f9b-40a7-8670-c8bb247a28d7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.850322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 466/930] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:52.850322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3ab72-9f9b-40a7-8670-c8bb247a28d7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:52.857709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae993a21-fb9f-4139-87a9-5fcc0a1e1bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:52.858759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae993a21-fb9f-4139-87a9-5fcc0a1e1bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.859042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae993a21-fb9f-4139-87a9-5fcc0a1e1bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.859314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae993a21-fb9f-4139-87a9-5fcc0a1e1bb1 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.860310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae993a21-fb9f-4139-87a9-5fcc0a1e1bb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:52.862021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae993a21-fb9f-4139-87a9-5fcc0a1e1bb1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:52.880582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 465/931] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:03:52.881835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:52.882124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:52.882610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:52.911036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:52.913469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:52.942855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:52.969402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3ab72-9f9b-40a7-8670-c8bb247a28d7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.007202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.007808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.008083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.008400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.009055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:53.009428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.010784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.021370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 466/932] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:53.031222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 467/933] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:53.037402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.085886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.156163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4044a7-107d-4e33-b88d-04f49a3aff22 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.191651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.192307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.192585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.192851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.200985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.202364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-cd118b4f-bb33-4bf8-87b9-67424fec34ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:53.214316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:53.215315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.216929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 467/934] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:53.223755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.263215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ddfc3-607c-46e8-bf92-cbe1c77099dd None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.263485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ddfc3-607c-46e8-bf92-cbe1c77099dd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.263756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ddfc3-607c-46e8-bf92-cbe1c77099dd None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.268213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 468/935] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:53.270166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.270442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.270705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.290460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:53.291646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:53.343626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d519c-69ff-411e-a9b1-6c16188ca821 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.364256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 469/936] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:53.365080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.388520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.389626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.390111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.390597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.391641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:53.392147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.393980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.398021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.407586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.417482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 468/937] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:53.427520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.479761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c96fe-d76e-4e35-b60f-62b75358e46b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.519831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.520813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.521463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.521860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.522592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.524637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.528111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING keystone.server.flask.common [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 11:03:53.537462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789fdc8-5801-488d-9a17-a304df1793ce None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.554264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.sql.core [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 11:03:53.554264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 11:03:53.554264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [parameters: {'id': '846542dc43764001ab6dbec00da128b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 11:03:53.554264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 846542dc43764001ab6dbec00da128b2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 846542dc43764001ab6dbec00da128b2. Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 11:03:53.564143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 18 11:03:53.565069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application yield resource Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise self.value Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application session.commit() Aug 18 11:03:53.566030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.flush() Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application compat.raise_( Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 11:03:53.567684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 11:03:53.568832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 11:03:53.569810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [parameters: {'id': '846542dc43764001ab6dbec00da128b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.577298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:53.578526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 846542dc43764001ab6dbec00da128b2. Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 470/938] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Aug 18 11:03:53 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:53.579768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.589628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.590416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.591598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.592104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.606051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.609283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:53.629070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:53.631088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:53.632808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:53.632808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:53.632808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:53.632808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 469/939] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:53.636869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.679192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.693328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.699486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dd195-2b94-4935-922c-6a9823c1eccf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.736171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.736789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.737061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.737335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.743847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.745512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.746892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb3f6b-c294-4f33-8520-ac87a6bfba05 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:53.759190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:53.760115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.761056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 471/940] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:53.761587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.785221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.785870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.786152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.786420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.787056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:53.787356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.788751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.803922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 470/941] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:53.804763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.873416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc17695-d432-4bf1-b349-b99c19fb1127 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.910627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.911258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.911535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.911803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.912410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.912850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28d36f-3c60-406d-8586-7a79ba5f0e54 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:53.913776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:53.923054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:53.924108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:53.925056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 472/942] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:53.925860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:53.930373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.937972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:53.959624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:53.960265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:53.960549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:53.960827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:53.961493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:53.961798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:53.963349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:53.981926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 471/943] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:53.982880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:54.057429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486762-a601-4f43-ae46-a4444ee4e8aa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:54.100622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:54.101270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.101549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.101823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.102685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84eb07-ba65-4fea-bf35-fa4dac3372bd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:54.110098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:54.111710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:54.125719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:54.126838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:54.127867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:54.131689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 473/944] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:54.132636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:54.144460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:54.145652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.146077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.146513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.147600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:54.147904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:54.149325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:54.155486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 472/945] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:54.161035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:54.203843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:54.226595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:54.269802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388e907-9c4a-42b5-805f-7fc97a042cba None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:54.290989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17592b2-e491-43bb-b16b-5db8f7e49a75 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:54.308859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:54.310333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.310835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.311319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.312231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:54.312764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:54.314811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:54.331857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 474/946] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:54.338174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:54.338317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:54.338787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.339065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.339349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.340056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:54.340399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:54.342599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:54.349564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 473/947] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:54.355617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:54.472067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:54.476656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254c1b2-e7f3-43f5-b2dd-df3d7f8cbd92 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:54.478599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980642c-fba4-4f9e-a81b-070848703111 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:54.515137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dcb77-61b0-47eb-861d-b90bda3feaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:54.515960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcb77-61b0-47eb-861d-b90bda3feaa9 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.516261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcb77-61b0-47eb-861d-b90bda3feaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.516535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcb77-61b0-47eb-861d-b90bda3feaa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.517205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dcb77-61b0-47eb-861d-b90bda3feaa9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:54.518573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dcb77-61b0-47eb-861d-b90bda3feaa9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:54.519361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:54.531779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fff411-7233-4217-83d5-8c728716d145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:54.532857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fff411-7233-4217-83d5-8c728716d145 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.533338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fff411-7233-4217-83d5-8c728716d145 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.533899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fff411-7233-4217-83d5-8c728716d145 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/25f03b43ee624965b53e9e3163fbeb13/roles/6cfa4326e893449aa1e4daa24e4223e7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.537244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fff411-7233-4217-83d5-8c728716d145 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=25f03b43ee624965b53e9e3163fbeb13, role_id=6cfa4326e893449aa1e4daa24e4223e7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:54.539047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fff411-7233-4217-83d5-8c728716d145 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:54.547316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 474/948] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:03:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:54.557575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 475/949] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:03:54 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/25f03b43ee624965b53e9e3163fbeb13/roles/6cfa4326e893449aa1e4daa24e4223e7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:03:54.956577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8233-ae5a-48c2-af0a-3bf0711860bd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.957109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8233-ae5a-48c2-af0a-3bf0711860bd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.957109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8233-ae5a-48c2-af0a-3bf0711860bd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.958727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 475/950] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:54.964174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:54.964604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:54.964780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:54.983644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:54.985216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:55.072307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 476/951] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:55.338723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8233-ae5a-48c2-af0a-3bf0711860bd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:55.448299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8233-ae5a-48c2-af0a-3bf0711860bd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:55.484623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:55.485605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:55.486118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:55.486589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:55.487425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:55.489022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:55.498569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:55.499581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:55.500483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:55.503164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 476/952] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:55.510087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:55.621246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb6b34-0254-414b-86e1-27176b638712 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:55.637046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:55.659825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:55.660902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:55.661493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:55.662170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:55.662811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:55.663114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:55.664511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:55.674673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 477/953] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:55.676099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:55.702135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:55.793922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d1a2d-a0ba-4b4b-825d-369c5d2386ec None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:55.837110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:55.837712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:55.837993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:55.838264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:55.838897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:55.839204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:55.840824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8725f-9c0b-428c-a8ca-c2dd13f0f67d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:55.850583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 477/954] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:55.851909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:55.957455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb0545-4f1b-4118-83d8-150a3182a7bd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:55.993519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e5399-35ea-496f-9e53-5cf8568d4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:55.994157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e5399-35ea-496f-9e53-5cf8568d4ddd None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:55.994386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e5399-35ea-496f-9e53-5cf8568d4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:55.994856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e5399-35ea-496f-9e53-5cf8568d4ddd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:55.995670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e5399-35ea-496f-9e53-5cf8568d4ddd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:55.997433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e5399-35ea-496f-9e53-5cf8568d4ddd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:56.026710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 478/955] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:03:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:03:56.130134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1a8bd-8a15-4c52-891f-318684b05a87 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.130692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1a8bd-8a15-4c52-891f-318684b05a87 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.130692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1a8bd-8a15-4c52-891f-318684b05a87 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.132347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 478/956] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:56.138259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb202-4a11-4f0e-a2a2-dfc720d635a0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.138880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb202-4a11-4f0e-a2a2-dfc720d635a0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.139421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb202-4a11-4f0e-a2a2-dfc720d635a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.159612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-adebb202-4a11-4f0e-a2a2-dfc720d635a0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:56.161228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-adebb202-4a11-4f0e-a2a2-dfc720d635a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:56.192587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:56.244101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 479/957] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:56.513041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1a8bd-8a15-4c52-891f-318684b05a87 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:56.559260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe7473-515c-46eb-8bc0-cbcdd1373945 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.559920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe7473-515c-46eb-8bc0-cbcdd1373945 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.560494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe7473-515c-46eb-8bc0-cbcdd1373945 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.562307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 480/958] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:56.563983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80162-5d5a-4ea9-9322-6c448ecf69ba None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.564548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80162-5d5a-4ea9-9322-6c448ecf69ba None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.565031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80162-5d5a-4ea9-9322-6c448ecf69ba None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.566716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 481/959] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:03:56.567944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06424356-3e19-4297-9d0c-2dd03e87f99b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.568454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06424356-3e19-4297-9d0c-2dd03e87f99b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.568942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06424356-3e19-4297-9d0c-2dd03e87f99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.587939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-06424356-3e19-4297-9d0c-2dd03e87f99b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:56.589092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-06424356-3e19-4297-9d0c-2dd03e87f99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:56.609548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:56.631236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1a8bd-8a15-4c52-891f-318684b05a87 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:56.662056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 482/960] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:56.663570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.663848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.664118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.676774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:56.677701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.678225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.678741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.679636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:56.681255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:56.683950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING keystone.server.flask.common [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 11:03:56.685968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:56.687342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:56.697722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 11:03:56.697722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 11:03:56.697722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [parameters: {'user_id': 'd921b08ffc7a493ea1693b7d5cd6df18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 11:03:56.697722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Aug 18 11:03:56.716872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 11:03:56.718055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 11:03:56.719336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 11:03:56.720477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 11:03:56.721656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 11:03:56.722756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd921b08ffc7a493ea1693b7d5cd6df18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:56.723914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 18 11:03:56.724987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 18 11:03:56.727082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 18 11:03:56.727082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 18 11:03:56.727082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 11:03:56.727082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 11:03:56.727082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 11:03:56.727082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:56.731002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 479/961] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Aug 18 11:03:56 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:56.732512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:56.779514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 483/962] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:56.846651 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb7a75-7c83-4408-a77b-bfe4d14dd528 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:56.880833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:56.881692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:56.882128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:56.882543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:56.890985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:56.892856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:56.904819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:56.905927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 11:03:56.907118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:56.909243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 480/963] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:03:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:56.917427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:56.940012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:56.963319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:56.968640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.037970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac799e-9e96-4d58-9e93-5b11e2197a9b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.073501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19893286-e9cb-4c42-9dc8-3b649a4a351e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.073704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.074386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.074666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.075476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.076326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:57.076633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.078063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.110708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.111337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.111615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.111887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.115253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 484/964] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Aug 18 11:03:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:57.116323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.136532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:03:57.143268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:57.144546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:03:57.145580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:57.147614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 481/965] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Aug 18 11:03:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:57.153399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.220321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.227277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.251431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ebb7b-16bc-4ba5-b3ef-265c1bd10e2b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.274187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76df0-6b9e-41e9-8fb6-21a2f8af9a20 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.301669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.302862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.304033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.304713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.305754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.308794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.318249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.318855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.319132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:57.319350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:57.320496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:03:57.321364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:57.322121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.322121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:57.322121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.322327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 485/966] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:03:57.322397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.329344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.333653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 482/967] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:57.339337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.464989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abde9a6-4109-4f48-bf54-ea6366088653 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.470124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e410c74-ecff-49ca-9a82-db1d53584bd6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.497353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.501635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.502321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.502719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.503104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.503844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:57.504285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.505870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.507519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.508454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.508735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.509008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.516273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 486/968] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:57.517168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.524932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.526592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.527817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:57.546709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:57.547741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:57.548689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:57.548689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:57.548689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:57.548689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 483/969] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:57.553818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.631018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f296bc-0777-455e-b1eb-4ee6e91da8b9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.655377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ae0ed-1eb9-4a8f-ac07-1c6a88e762c8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.666763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.667676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.668152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.668729 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.669582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:57.669948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.671305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.675799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 487/970] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:57.690352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.690974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.691251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.691518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.692176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:57.692518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.693927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.708083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 484/971] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:57.713686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.795148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.832497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a4e8-874c-47b4-be95-829c642a890d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:57.867641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:57.868304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.868584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.868849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.876693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:57.878133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-16dab8eb-8730-4e6d-8bdb-1db31554d7ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:03:57.889260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:57.890226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 11:03:57.891201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:57.892316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 488/972] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:57.896423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:57.939100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:57.983788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71869a8-2de1-4932-892a-df614deef7ff None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.984095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71869a8-2de1-4932-892a-df614deef7ff None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.984387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71869a8-2de1-4932-892a-df614deef7ff None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:57.985874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 489/973] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:57.990676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86a479-c002-4440-8982-31d740135211 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:57.990964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86a479-c002-4440-8982-31d740135211 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:57.991224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86a479-c002-4440-8982-31d740135211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.008411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaecb3-b4aa-4287-824e-6e80fd256224 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.009948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ea86a479-c002-4440-8982-31d740135211 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:58.011156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea86a479-c002-4440-8982-31d740135211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:58.047929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.048582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.048863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.049137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.049872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:58.050177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.051610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.074666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 485/974] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Aug 18 11:03:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:58.076380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.088325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 490/975] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:03:58.175411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.182094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb6cc5-d346-48be-8bcc-de16762590d3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.218227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.218835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.219112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.219380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.287282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.288834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.351908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86a479-c002-4440-8982-31d740135211 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.392052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.400177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 486/976] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 11:03:58.407139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.413909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.456082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86a479-c002-4440-8982-31d740135211 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.492026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.492682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.492954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.493218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.493790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.495138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:58.503780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:58.504892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:03:58.505760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 491/977] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:58.510789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.515932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4cfc0-ccf6-4da3-b76f-dcdb400d283f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.551726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.552379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.552654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.552924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.562924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.564405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:03:58.579294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:58.580288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:03:58.581192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:03:58.581192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:03:58.581192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:58.584542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 487/978] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:58.585853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.622376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07817c2e-43ce-4787-aafb-f88ab71e5ea6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.640898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.646729 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.658960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.659565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.659842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.660108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.660779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:58.661079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.662438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.671781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 492/979] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:58.672923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.699028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805fa21-3c29-4ab7-87f8-a814fa30474a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.735366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.736008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.736270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.736662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.737307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:58.737610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.738969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.752460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 488/980] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:58.758216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.781358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c44bd6-6c90-47a9-94fd-960cd7e93e18 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.822091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.822742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.822979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.823254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.824045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:58.824383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.826857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.837196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 493/981] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:03:58.838169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.872887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a235fff-a613-4894-a9d6-96f98a9e2a6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.881978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.896808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:58.909315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.909939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.910215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.910484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.917609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.920775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:03:58.933212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:03:58.934096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:03:58.935106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:03:58.936382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 489/982] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:03:58.940622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:58.953101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97da7f-d20f-4510-89f9-c97257dce74e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:58.989357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:58.989952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:58.990237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:58.990510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:58.991205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] RBAC: Adding query filter params (interface=admin, service_id=e573cc03d9fb4244b06134fcbc17e47a, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:58.991514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:58.992943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:58.998473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 494/983] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/endpoints?service_id=e573cc03d9fb4244b06134fcbc17e47a&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:59.003949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:59.057572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3439cb-7eae-4bd1-8049-e010a373694b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:59.092693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:59.093292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:59.093574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:59.093845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:59.094541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:59.094845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:59.096327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:59.109530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 490/984] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:03:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:59.115594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:03:59.125607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5f922-bc48-49da-a029-443a60414c98 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:59.144058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:59.145863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:03:59.166413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9582c-045d-4984-8d35-9b94352b2f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:59.167030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9582c-045d-4984-8d35-9b94352b2f7a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:59.167312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9582c-045d-4984-8d35-9b94352b2f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:59.167594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9582c-045d-4984-8d35-9b94352b2f7a None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:59.168239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9582c-045d-4984-8d35-9b94352b2f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:59.169617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9582c-045d-4984-8d35-9b94352b2f7a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:59.175711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 495/985] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:03:59 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:03:59.233110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bcbf8-6d27-4903-8963-8419c3605818 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:03:59.272633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:03:59.273349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:59.273705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:59.274045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:59.274851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] RBAC: Adding query filter params (role.id=6cfa4326e893449aa1e4daa24e4223e7, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=25f03b43ee624965b53e9e3163fbeb13) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:03:59.275228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:03:59.276806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff5cae-f853-4c57-8e03-72400ac5ee44 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:03:59.284631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 491/986] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:03:59 2022] GET /identity/v3/role_assignments?user.id=25f03b43ee624965b53e9e3163fbeb13&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=6cfa4326e893449aa1e4daa24e4223e7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:03:59.611544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984b08-04f5-4428-bb79-ae6dca5335ff None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:59.611925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984b08-04f5-4428-bb79-ae6dca5335ff None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:59.612126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984b08-04f5-4428-bb79-ae6dca5335ff None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:59.613608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 496/987] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:03:59.618672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20d48d-9942-478c-80e9-1c07ef3d9d99 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:03:59.618962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20d48d-9942-478c-80e9-1c07ef3d9d99 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:03:59.619237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20d48d-9942-478c-80e9-1c07ef3d9d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:03:59.637949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5b20d48d-9942-478c-80e9-1c07ef3d9d99 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:03:59.639498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5b20d48d-9942-478c-80e9-1c07ef3d9d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:03:59.708299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 492/988] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:03:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:03:59.975664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984b08-04f5-4428-bb79-ae6dca5335ff None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:00.087609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984b08-04f5-4428-bb79-ae6dca5335ff None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:00.122794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a980-991c-4965-b89f-28a1f7accf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:00.123451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a980-991c-4965-b89f-28a1f7accf45 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:00.123733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a980-991c-4965-b89f-28a1f7accf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:00.124000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a980-991c-4965-b89f-28a1f7accf45 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:00.124679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a980-991c-4965-b89f-28a1f7accf45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:00.126043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a980-991c-4965-b89f-28a1f7accf45 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:00.141738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 497/989] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:03:59 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:01.109041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b6a0-1cf4-4a7e-a34f-e02eeb8c67ba None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.109566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b6a0-1cf4-4a7e-a34f-e02eeb8c67ba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.109669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b6a0-1cf4-4a7e-a34f-e02eeb8c67ba None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.111082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 493/990] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:01.116134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36d6b-0042-4ede-ab9e-850b1a08d7fa None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.116457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36d6b-0042-4ede-ab9e-850b1a08d7fa None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.116826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36d6b-0042-4ede-ab9e-850b1a08d7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.135314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bdd36d6b-0042-4ede-ab9e-850b1a08d7fa None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:01.136557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bdd36d6b-0042-4ede-ab9e-850b1a08d7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:01.174688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:01.217501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 498/991] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:01.238768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d730e7-76f0-4b37-bc06-c5f57f5aac0f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.239086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d730e7-76f0-4b37-bc06-c5f57f5aac0f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.239480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d730e7-76f0-4b37-bc06-c5f57f5aac0f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.241013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 494/992] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:01.247427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.247978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.248284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.266131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:01.267317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:01.334104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 499/993] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:01.490277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d730e7-76f0-4b37-bc06-c5f57f5aac0f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:01.515451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:01.598343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d730e7-76f0-4b37-bc06-c5f57f5aac0f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:01.605843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:01.637172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:01.637834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.638119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.638509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.639098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:01.640486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:01.650673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:01.651684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:01.652573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 495/994] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Aug 18 11:04:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:01.658574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:01.707597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:01.715057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191add6-09f5-472c-838b-2e2dd4b0bb21 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:01.750109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:01.753034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:01.753816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.753930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.754242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] PATH_INFO: `/v3/services/image` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.754823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:01.756488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:01.765871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:01.766970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:01.773058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:01.773835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064fe5-47ef-4e03-82a1-ff5d6c5e1be6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:01.773912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 500/995] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Aug 18 11:04:01 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:04:01.773912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:01.815273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:01.815881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.816162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.816485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.817118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:01.817462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:01.818849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:01.824387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 496/996] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:04:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:01.830189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:01.879269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ba343-4b4c-4406-a5cf-02dc43f1ba76 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:01.916672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:01.917278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.917553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.917827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.918678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] RBAC: Adding query filter params (name=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:01.919591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:01.920987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:01.928240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 501/997] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:01.932399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:01.938529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5a1c-b82e-410a-8848-78eb26a4f2cf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:01.962909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:01.977591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:01.978203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:01.978898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:01.979179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:01.979821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:01.980129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:01.982224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:01.988174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 497/998] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Aug 18 11:04:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:01.993558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:02.010877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:02.053352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aaf2-2003-4613-9761-d3d6df0a065c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:02.089840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:02.090446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.091394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.091716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.092366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:02.092670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:02.094033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303024c-00dd-4b53-ae47-1e0c87009df6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:02.098500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 502/999] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:02.099675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99aaac-8995-4d01-b8eb-8663c74b7039 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.099970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99aaac-8995-4d01-b8eb-8663c74b7039 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.100255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99aaac-8995-4d01-b8eb-8663c74b7039 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.101474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 503/1000] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:02.107393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.107698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.107956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.111787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd6bb4-cbd8-4422-bdb3-90d2a59ca509 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:02.130254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:02.132024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:02.151027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:02.151662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.151948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.152250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.152942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:02.154907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:02.175049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 498/1001] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:04:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:04:02.226418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 504/1002] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:02.500372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:02.580917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:02.608888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f34c8c-9587-44aa-ae16-b481c48a6804 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:02.642347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:02.642974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.643247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.643512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.644085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:02.645485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:02.655517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:02.656628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:04:02.657483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:02.660553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 499/1003] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:04:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:02.662445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:02.677933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:02.764867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09ad4-d736-4207-97b3-6eda313644a8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:02.798357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:02.798957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.799229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.799494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.800113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:02.800439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:02.801782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:02.811769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 505/1004] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:04:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:02.812951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:02.911592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ad96-cf49-4c94-8649-0f66ee2fb551 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:02.945015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:02.945622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:02.945900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:02.946173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:02.946801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:02.947104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:02.948502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:02.958555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 500/1005] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:04:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:02.959541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:03.046019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:03.069380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7a307-1ed7-44ca-93b5-0533325d7484 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:03.105161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:03.105766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:03.106045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:03.106385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:03.107233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] RBAC: Adding query filter params (interface=public, service_id=4cc59c4ea6db4dcb84234029c37cac2e, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:03.107541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:03.108983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35178e9-2132-4e9a-8dc8-0fbc34a20b18 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:03.118780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 506/1006] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:04:02 2022] GET /identity/v3/endpoints?service_id=4cc59c4ea6db4dcb84234029c37cac2e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:03.119904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:03.123946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:03.226534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23d3c-a179-4905-9430-b4be8ecda874 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:03.260475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1d54b-7cae-4e8d-9ad9-1f1ba47f5ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:03.261083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1d54b-7cae-4e8d-9ad9-1f1ba47f5ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:03.261362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1d54b-7cae-4e8d-9ad9-1f1ba47f5ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:03.261633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1d54b-7cae-4e8d-9ad9-1f1ba47f5ad6 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:03.262237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1d54b-7cae-4e8d-9ad9-1f1ba47f5ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:03.264218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1d54b-7cae-4e8d-9ad9-1f1ba47f5ad6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:03.276501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 501/1007] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:04:03 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:04:04.048045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecba27-b224-48b9-9699-5ca3791eefca None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.048585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecba27-b224-48b9-9699-5ca3791eefca None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.048664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecba27-b224-48b9-9699-5ca3791eefca None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.050336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 507/1008] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:04.056115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.056445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.056728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.075901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:04.077142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:04.126221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37b819-6695-4ceb-89fb-4ab72b5f19dd None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.126503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37b819-6695-4ceb-89fb-4ab72b5f19dd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.126778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37b819-6695-4ceb-89fb-4ab72b5f19dd None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.128265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 508/1009] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:04:04.133063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.133349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.133624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.141073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:04.149636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 502/1010] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:04.152960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:04.154192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:04.220495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 509/1011] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:04.421853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:04.491365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:04.524657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c5587-6a14-4542-8c12-a4c5967386c8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:04.560415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:04.563440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:04.564304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.564592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.564867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.565520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:04.567647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:04.584232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 503/1012] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:04:04 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:04.591954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22879f4c-47c9-4f07-b8c5-22cc8d963ce6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:04.637868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e46ddc-557c-4be3-9278-2bde5b9accdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:04.638529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e46ddc-557c-4be3-9278-2bde5b9accdf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.638808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e46ddc-557c-4be3-9278-2bde5b9accdf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.639082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e46ddc-557c-4be3-9278-2bde5b9accdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.640321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e46ddc-557c-4be3-9278-2bde5b9accdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:04.641771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e46ddc-557c-4be3-9278-2bde5b9accdf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:04.644239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 510/1013] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:04.649429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:04.713022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:04.767513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dca502-ec57-4b01-9f69-0d98209d9745 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:04.808688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7c579-c46e-430e-a9b0-e93ab39792cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:04.809344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7c579-c46e-430e-a9b0-e93ab39792cc None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:04.809631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7c579-c46e-430e-a9b0-e93ab39792cc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:04.809975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7c579-c46e-430e-a9b0-e93ab39792cc None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:04.810677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7c579-c46e-430e-a9b0-e93ab39792cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:04.812568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7c579-c46e-430e-a9b0-e93ab39792cc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:04.843575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 504/1014] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:04:04 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:05.208798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae3008-bdcc-4c6b-8ea8-f278b959ebd3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:05.209321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae3008-bdcc-4c6b-8ea8-f278b959ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:05.209321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae3008-bdcc-4c6b-8ea8-f278b959ebd3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:05.210859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 511/1015] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:05.216405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:05.216894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:05.216970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:05.234726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:05.235918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:05.307080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 505/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:05 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:05.577018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae3008-bdcc-4c6b-8ea8-f278b959ebd3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:05.679091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae3008-bdcc-4c6b-8ea8-f278b959ebd3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:05.715344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:05.716041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:05.716348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:05.716619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:05.717192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:05.718548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:05.721198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:05.728879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:05.729843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 11:04:05.730762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:05.735930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 512/1017] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:04:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:05.737343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:05.777374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:05.840735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e3adc-0eff-4646-8410-9c6da4d181fb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:05.876378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:05.877289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:05.877745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:05.878218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:05.879020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:05.879508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:05.881073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:05.886044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 506/1018] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:04:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:05.891870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:05.995706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5d3ae-fa6a-4fe5-8da0-37d31bbb1b06 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:06.033547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:06.034390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:06.034863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:06.035312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:06.036129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:06.036635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:06.038223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75a62-364b-4749-8a59-131fb2be0978 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:06.048707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 513/1019] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Aug 18 11:04:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:06.049921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:06.155590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c7847-1ec1-4303-8282-a93e33c163f0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:06.157277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:06.193291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954d0d5-be0a-41bc-8e2b-7b558c91031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:06.193916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954d0d5-be0a-41bc-8e2b-7b558c91031e None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:06.194197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954d0d5-be0a-41bc-8e2b-7b558c91031e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:06.194471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954d0d5-be0a-41bc-8e2b-7b558c91031e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:06.195208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954d0d5-be0a-41bc-8e2b-7b558c91031e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:06.196606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954d0d5-be0a-41bc-8e2b-7b558c91031e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:06.224218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 507/1020] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:04:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:04:06.534201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac412-139b-4cf3-aef2-1ae533fbc112 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:06.534765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac412-139b-4cf3-aef2-1ae533fbc112 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:06.534765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac412-139b-4cf3-aef2-1ae533fbc112 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:06.536081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 514/1021] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:06.541216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08972-6dd1-4500-a6cf-3252e79042d6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:06.541501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08972-6dd1-4500-a6cf-3252e79042d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:06.541777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08972-6dd1-4500-a6cf-3252e79042d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:06.559735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68e08972-6dd1-4500-a6cf-3252e79042d6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:06.561034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68e08972-6dd1-4500-a6cf-3252e79042d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:06.634441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 508/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:06.826834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb364a-862f-4542-a411-45151521e579 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:06.827024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb364a-862f-4542-a411-45151521e579 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:06.827305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb364a-862f-4542-a411-45151521e579 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:06.829229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 515/1023] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:06.836154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:06.836479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:06.836759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:06.858049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:06.859242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:06.905928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb364a-862f-4542-a411-45151521e579 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:06.931049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 509/1024] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:06.973652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:07.016046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb364a-862f-4542-a411-45151521e579 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.052840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.053459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.053741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.054011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] PATH_INFO: `/v3/services/image` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.054581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.055953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:07.065402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:07.066376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:07.067302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 516/1025] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Aug 18 11:04:06 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:04:07.072536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:07.088917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:07.179232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401dd28-a0a6-4689-a093-7ae35f98539d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.213779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:07.223298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.223911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.224181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.224515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.225087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] RBAC: Adding query filter params (name=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:07.225390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.226805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.236965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 510/1026] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:07.238278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:07.327480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fb197-287d-40ec-9d16-60b95b146144 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.331537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:07.355771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4eee2-cf91-41d4-b13f-295ca04bf97d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.356995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:07.365450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.366078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.366358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.366633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.367732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.369193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.370917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 517/1027] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:07.376920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:07.396297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.396927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.397209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.397483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.398136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:07.398439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.399863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.406258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 511/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:07.412116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:07.489932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020c3a9-3280-4239-b03b-4e55ae7341d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.521517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a7946-8040-4b0e-9ed0-5478ae20bb27 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.528623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.529212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.530245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.530737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.531994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.534395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.568229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 518/1029] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:04:07 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:04:07.568463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.568463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.568767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.569043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.569763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] RBAC: Adding query filter params (interface=public, service_id=1af89641170b43db908a77a2c9f48ecf, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:07.570074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.571568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fbf36-99fb-48cd-8786-49c61ae89c14 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.580242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 512/1030] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:04:07 2022] GET /identity/v3/endpoints?service_id=1af89641170b43db908a77a2c9f48ecf&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:07.583715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:07.646568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:07.701932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a20c-0e93-434f-91e7-e10676f44e90 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:07.740376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab71a5-5579-4f13-9ac4-d57e313748ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:07.740991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab71a5-5579-4f13-9ac4-d57e313748ba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:07.741268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab71a5-5579-4f13-9ac4-d57e313748ba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:07.741546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab71a5-5579-4f13-9ac4-d57e313748ba None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:07.742183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab71a5-5579-4f13-9ac4-d57e313748ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:07.743578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab71a5-5579-4f13-9ac4-d57e313748ba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:07.750786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 519/1031] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:04:07 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:04:08.259196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ffe8-6252-41dd-8de0-bb5c1724530b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:08.259723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ffe8-6252-41dd-8de0-bb5c1724530b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:08.259723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ffe8-6252-41dd-8de0-bb5c1724530b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:08.261262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 513/1032] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:08.266876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:08.267170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:08.267447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:08.286356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:08.287484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:08.359122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 520/1033] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:08.629366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ffe8-6252-41dd-8de0-bb5c1724530b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:08.638527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:08.738182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ffe8-6252-41dd-8de0-bb5c1724530b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:08.774182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:08.774834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:08.775113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:08.775382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:08.786380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:08.791067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:08.792104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:08.793024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 514/1034] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Aug 18 11:04:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:08.798355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:08.848716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:08.910563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055c7b-a674-48ac-9c32-abf2181bb8b6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:08.944390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:08.944991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:08.945269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:08.945543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:08.946212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:08.946499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:08.947850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:08.956639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 521/1035] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:04:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:08.962161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.036508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:09.071973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec424ef0-ecb3-4671-8fb6-94498e08314a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:09.107026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:09.107654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.107937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.108232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.118523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:09.119973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:09.135647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:09.136713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:09.137667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:09.137667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:09.137667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:09.140830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 515/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:09.143910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.245073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce151d5-e079-4432-905d-4acd4ff21ab8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:09.255994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:09.282585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:09.283190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.283469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.283738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.284397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:09.284699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:09.286054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:09.302663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 522/1037] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:09.303967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.407192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa37a8-aabb-445e-8332-fb1b02244eff None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:09.424339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:09.443271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:09.443899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.444181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.444479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.452894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:09.454329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4110dfdc-b07d-4fd5-9231-129fe778c94d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:04:09.465402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:09.466359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:09.473910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 516/1038] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:04:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:09.474651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.567641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5919-8f25-4a9b-97d8-dbfb86d7ec58 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.567946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5919-8f25-4a9b-97d8-dbfb86d7ec58 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.568111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5919-8f25-4a9b-97d8-dbfb86d7ec58 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.569556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 517/1039] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:04:09.574553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad7b67-28cf-4b45-b2b6-b2405ccad229 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.574863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad7b67-28cf-4b45-b2b6-b2405ccad229 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.575144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad7b67-28cf-4b45-b2b6-b2405ccad229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.578227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1a69b-9375-4d83-ab43-25b03c9749fa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:09.594580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-50ad7b67-28cf-4b45-b2b6-b2405ccad229 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:09.595865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-50ad7b67-28cf-4b45-b2b6-b2405ccad229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:09.614533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:09.615135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.615418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.615687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.617224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:09.617224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:09.619106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:09.642104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 523/1040] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:04:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:09.643145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.670557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:09.671864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 518/1041] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:09.698808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f782af-e3c3-4783-a60a-58421079ab83 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.699086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f782af-e3c3-4783-a60a-58421079ab83 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.699357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f782af-e3c3-4783-a60a-58421079ab83 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.701007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 519/1042] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:04:09.705706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.705996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.706359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.724064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:09.725343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:09.753850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506364c7-9bd4-48b9-8f9e-a88c3120af82 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:09.790066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:09.790671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:09.790952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:09.791234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:09.806056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 520/1043] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:09.865057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:09.866469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:09.892946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:09.941824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.978696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 524/1044] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 11:04:09.979623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:09.983226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.055911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9352-6176-4cc9-b6a9-d63d0914ce2c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.090327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04f82f-e763-48ad-a1f2-adee79bb49ac None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.093101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.093731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.094015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.094291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.104642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:10.110052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:10.110985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:10.111839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 521/1045] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:04:10.112580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.118381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.127579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.128226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.128506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.128777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.139098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.140580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:10.155535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:10.156501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:10.157402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:10.157402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:10.157402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:10.157402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 525/1046] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:10.157634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.214378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.219176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a04cb3-9942-4319-9a8d-7b82f188e7b6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.255820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.256475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.256760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.257031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.257614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.258987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.263350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f30f-c80f-4c9e-8050-15319350e85a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:10.268024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:10.269124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:10.269979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 522/1047] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:10.270783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.297690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.298287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.298562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.298835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.299484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:10.299783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.301174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.310522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 526/1048] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:10.311160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.355566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.375035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840f976-1cd3-4a4b-9386-fa272fe90fc2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.408758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.409437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.409726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.409992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.410630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:10.410954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.412344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.415112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcb16-eb77-407b-8e5d-5eadd83a6c15 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.423927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 523/1049] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:10.424761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.447962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.455223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.455830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.456109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.456426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.457047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:10.457348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.458699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.463445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 527/1050] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:10.464229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.526510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545f460-af75-4450-8644-b2abd10d0a2d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.565383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.566011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.566288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.566569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] PATH_INFO: `/v3/users/ec876f0814da42b8a9ee1a0ffdaf964d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.573015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964edd-ddb0-4ac6-a8d1-1827fc2e03f9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.575927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.581146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] RBAC: Authorizing `identity:get_user(user_id=ec876f0814da42b8a9ee1a0ffdaf964d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.582589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.584251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 524/1051] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/users/ec876f0814da42b8a9ee1a0ffdaf964d => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:10.585020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.611199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.611825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.612113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.612422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.618992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.620450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:04:10.632158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:10.633245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 528/1052] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:10.634363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.669418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.688896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e988e6b-de71-48c5-8745-42020ca746fc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.722536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.723130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.723409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.723681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.724337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:10.724637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.725993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.730627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 525/1053] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:10.731430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.734889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa195730-2cd1-4b2b-a035-b014c465a8f1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.770597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.771196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.771473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.771745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.801029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.835082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe460b3f-39c1-46d5-99ee-49fadb90ea0d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:10.838747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.840209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.873824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:10.874423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:10.874701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:10.874972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:10.875661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:10.875962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:10.877422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:10.889679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 526/1054] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:10.890392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.895794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:10.941589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 529/1055] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:10.942512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:10.993505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12e1e8-c885-49d7-b501-819a995e4047 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:11.008834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:11.030239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:11.030850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:11.031130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:11.031402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:11.032062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:11.033465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:11.048642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d97f-c320-4f00-9246-ccb39a974105 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:11.055280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 527/1056] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:04:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:11.057050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:11.097938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:11.098555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:11.098833 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:11.099108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:11.099847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] RBAC: Adding query filter params (role.id=6cfa4326e893449aa1e4daa24e4223e7, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=1f43ff51c5f84d34978c46bce6db67ee) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:11.100145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:11.101687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:11.109082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 530/1057] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:10 2022] GET /identity/v3/role_assignments?user.id=1f43ff51c5f84d34978c46bce6db67ee&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=6cfa4326e893449aa1e4daa24e4223e7 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:11.130301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:11.185398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa814f0-9653-4f76-b448-8108c6175765 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:11.226418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57540669-5de0-430c-b3b7-76162b6e1b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:11.227165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57540669-5de0-430c-b3b7-76162b6e1b21 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:11.227455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57540669-5de0-430c-b3b7-76162b6e1b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:11.227737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57540669-5de0-430c-b3b7-76162b6e1b21 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:11.229004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57540669-5de0-430c-b3b7-76162b6e1b21 None admin] RBAC: Authorizing `identity:get_project(project_id=d15133acb06044b1b028ebc5e29ccb9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:11.230484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57540669-5de0-430c-b3b7-76162b6e1b21 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:11.233322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 528/1058] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:11 2022] GET /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:11.239387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:11.349576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e8b0f-c2e9-4f90-98d7-a1cabe23c704 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:11.373781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:11.385660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:11.386264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:11.386542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:11.386810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:11.387812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=d15133acb06044b1b028ebc5e29ccb9e, user.id=ec876f0814da42b8a9ee1a0ffdaf964d) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:11.388124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:11.389564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ebb09-afdd-46f0-b606-418555ba27fb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:11.395204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 531/1059] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:11 2022] GET /identity/v3/role_assignments?user.id=ec876f0814da42b8a9ee1a0ffdaf964d&scope.project.id=d15133acb06044b1b028ebc5e29ccb9e&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:13.006825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c29a18-a9b6-4d2d-94c8-a2f01c1a4380 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.007388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c29a18-a9b6-4d2d-94c8-a2f01c1a4380 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.007388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c29a18-a9b6-4d2d-94c8-a2f01c1a4380 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.008739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 529/1060] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:13.014484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a808-1e50-4bf9-aca9-89b7d8f8e58a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.014895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a808-1e50-4bf9-aca9-89b7d8f8e58a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.015095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a808-1e50-4bf9-aca9-89b7d8f8e58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.029338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bab81-2408-410b-b368-fb1c026dce3f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.029620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bab81-2408-410b-b368-fb1c026dce3f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.029894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bab81-2408-410b-b368-fb1c026dce3f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.031369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 530/1061] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:04:13.033589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f174a808-1e50-4bf9-aca9-89b7d8f8e58a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:13.034790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f174a808-1e50-4bf9-aca9-89b7d8f8e58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:13.036227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f49ef-3bc6-481a-9d58-dfc34a51769b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.036512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f49ef-3bc6-481a-9d58-dfc34a51769b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.036787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f49ef-3bc6-481a-9d58-dfc34a51769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.055277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-182f49ef-3bc6-481a-9d58-dfc34a51769b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:13.056513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-182f49ef-3bc6-481a-9d58-dfc34a51769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:13.108367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 532/1062] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:13.132570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 531/1063] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:13.302104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d1b5-fe7e-490b-b5ae-a3da098ad191 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.302320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d1b5-fe7e-490b-b5ae-a3da098ad191 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.304009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d1b5-fe7e-490b-b5ae-a3da098ad191 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.304009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 533/1064] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:13.309185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.309636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.310076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.328821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:13.329995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:13.376797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d1b5-fe7e-490b-b5ae-a3da098ad191 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.386209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:13.402427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 532/1065] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:13.403317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.491602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d1b5-fe7e-490b-b5ae-a3da098ad191 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:13.510196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fad12e-2650-4bc5-817e-9ef24eb32243 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:13.511395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:13.531320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:13.532039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.532345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.532617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.545664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:13.546287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.546562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.546830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.557386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:13.562265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:13.563232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:13.564087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 533/1066] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:13.571221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.607379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:13.608850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:13.622337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:13.675514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d70970-c4a2-48ee-9f06-b4a0d371b813 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:13.710129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 534/1067] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 11:04:13.711021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.711267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:13.711792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.712072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.712377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.713042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:13.713351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:13.714724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:13.727579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 534/1068] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:13.727579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.730434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:13.811335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3f894-1050-4b61-9829-9918f2db0ba5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:13.821361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c63cb3-545a-4bdd-a9fa-ea7e4305fd84 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:13.832042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:13.847993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:13.848700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.848981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.849252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.859335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:13.859500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:13.859954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:13.860270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:13.860546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:13.864164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:13.865303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 535/1069] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:13.866149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.873344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:13.874788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:13.889360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:13.890388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:13.891299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:13.891299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:13.891299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:13.891299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 535/1070] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:13.891299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:13.951823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:13.973033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dc144-097f-432d-89e8-5f64dbb09b69 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:13.994156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7e9f-26ce-47fe-9edc-d37025018d10 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.008955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.009579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.009856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.010124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.020277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.021707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.031454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.032051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.032363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.032629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.033283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:14.033580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.034920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:14.037334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:14.038398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:14.039654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:14.039654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:14.039654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:14.039654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 536/1071] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:14.039654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.049200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 536/1072] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:14.050017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.078075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:14.143378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e849e-07a2-498d-8e67-85524ed56114 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.152571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fafb9-7cc0-4c2a-961f-901eb1c1d780 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.172289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:14.178417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.179020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.179300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.179573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.180236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:14.180544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.181917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.186956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.187553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.187831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.188101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.188776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:14.189077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.190426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.195728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 537/1073] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:14.196700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.206689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 537/1074] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:14.209042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.303162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:14.315452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc5b2-d2c0-4d97-a791-f06fb861d837 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.322619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4d53-da40-4bb5-a7a4-1986897c6fbf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.353966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.354516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.354788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.355055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] PATH_INFO: `/v3/users/ec876f0814da42b8a9ee1a0ffdaf964d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.356944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ec876f0814da42b8a9ee1a0ffdaf964d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.358373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.360122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 538/1075] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/users/ec876f0814da42b8a9ee1a0ffdaf964d => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:14.362229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.362866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.363145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.363415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.365906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.372497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.374203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:04:14.386445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:14.388017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:14.389041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 538/1076] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:04:14.393861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.441279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:14.491482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4db3-0d30-423f-a541-66d31187e4d3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.511209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2acb0-e4a7-48d9-ae4f-d6e522673936 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.525617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.526239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.526518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.526788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.528444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] RBAC: Authorizing `identity:get_project(project_id=d15133acb06044b1b028ebc5e29ccb9e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.529871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.531491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 539/1077] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:14.537311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.546155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.546763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.547045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.547319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.548049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:14.548405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.549838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.567499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 539/1078] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:14.573038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.575587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:14.636603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cd9e1-448e-44ab-ae0e-d2f94be6b930 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.662508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:14.672572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.672725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffc917-faba-4201-b3ef-4f265eaa8dfb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.673222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.673501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.673771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e/users/ec876f0814da42b8a9ee1a0ffdaf964d/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.675669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] RBAC: Authorizing `identity:create_grant(project_id=d15133acb06044b1b028ebc5e29ccb9e, user_id=ec876f0814da42b8a9ee1a0ffdaf964d, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.677130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.689626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 540/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:04:14 2022] PUT /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e/users/ec876f0814da42b8a9ee1a0ffdaf964d/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:04:14.708001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.708699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.708976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.709245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.721250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.722729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:14.739270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:14.740419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:14.741681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:14.741681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:14.741681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:14.741681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 540/1080] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:04:14.746583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.849624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f78622-0bb4-49cd-9c5a-7dff99a9bf64 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:14.882739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:14.883335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:14.883608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:14.883872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:14.884533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:14.884832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:14.886214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:14.897787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 541/1081] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:14.904345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:14.958574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:15.004342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4a15f-f656-4afc-9f27-60b08eec4250 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:15.038699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:15.039317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:15.039591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:15.039856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:15.046591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:15.048054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:04:15.058871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:15.059779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:15.060715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 541/1082] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:15.065940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:15.113576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:15.168282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525556e-2eb2-40f3-8767-61244795414d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:15.201479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:15.202066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:15.202338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:15.202604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:15.203276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:15.203573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:15.205021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b558b2-c082-463e-b588-25bb2b7da9f7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:15.216685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 542/1083] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:15.222719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:15.321601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e643982-2418-4c1a-bed6-558c41c863ed None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:15.335789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:15.356795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ea180-afc1-4d0f-9692-a179d1965051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:15.357440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ea180-afc1-4d0f-9692-a179d1965051 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:15.357714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ea180-afc1-4d0f-9692-a179d1965051 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:15.357981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ea180-afc1-4d0f-9692-a179d1965051 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/1f43ff51c5f84d34978c46bce6db67ee/roles/6cfa4326e893449aa1e4daa24e4223e7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:15.360079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ea180-afc1-4d0f-9692-a179d1965051 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=1f43ff51c5f84d34978c46bce6db67ee, role_id=6cfa4326e893449aa1e4daa24e4223e7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:15.361519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ea180-afc1-4d0f-9692-a179d1965051 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:15.374157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 542/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:04:15 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/1f43ff51c5f84d34978c46bce6db67ee/roles/6cfa4326e893449aa1e4daa24e4223e7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:04:16.593909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd3c0e-8c7d-43e3-b666-8bb01947509d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:16.594481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd3c0e-8c7d-43e3-b666-8bb01947509d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:16.594481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd3c0e-8c7d-43e3-b666-8bb01947509d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:16.595975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 543/1085] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:16.600960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:16.601390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:16.601510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:16.619312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:16.620535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:16.686769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 543/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:16.957906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd3c0e-8c7d-43e3-b666-8bb01947509d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.060917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd3c0e-8c7d-43e3-b666-8bb01947509d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.068016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.096358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728a822-a8fe-4de0-92f9-7c8b2ab51e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.096957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728a822-a8fe-4de0-92f9-7c8b2ab51e48 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.097236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728a822-a8fe-4de0-92f9-7c8b2ab51e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.097506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728a822-a8fe-4de0-92f9-7c8b2ab51e48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.107887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728a822-a8fe-4de0-92f9-7c8b2ab51e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c728a822-a8fe-4de0-92f9-7c8b2ab51e48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:17.112594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:17.113543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:17.114829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 544/1087] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:17.119663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.217937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677e0be-a22d-4fb6-b07f-96dffc2c09a9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.220149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.235435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddee73-072a-4db5-a012-469eb97def59 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.235715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddee73-072a-4db5-a012-469eb97def59 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.235988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddee73-072a-4db5-a012-469eb97def59 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.237382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 545/1088] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:04:17.242301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.242581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.242846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.255626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712a343-2409-4517-91f0-971022204acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.256262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712a343-2409-4517-91f0-971022204acd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.256546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712a343-2409-4517-91f0-971022204acd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.256819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712a343-2409-4517-91f0-971022204acd None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.257486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712a343-2409-4517-91f0-971022204acd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:17.257791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712a343-2409-4517-91f0-971022204acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:17.259194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712a343-2409-4517-91f0-971022204acd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:17.263088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:17.264310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:17.267919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 544/1089] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:17.273669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712a343-2409-4517-91f0-971022204acd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.332889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 546/1090] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:17.373755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712a343-2409-4517-91f0-971022204acd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.407108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.407723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.408002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.408309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] PATH_INFO: `/v3/users/ec876f0814da42b8a9ee1a0ffdaf964d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.409437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] RBAC: Authorizing `identity:get_user(user_id=ec876f0814da42b8a9ee1a0ffdaf964d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:17.410866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:17.412468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 545/1091] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/users/ec876f0814da42b8a9ee1a0ffdaf964d => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:17.418316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.509932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.516353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec4899-72de-434a-ae3f-2b29f961fbd6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.550447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.551048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.551327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.551599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.600890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.619026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:17.620498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:17.636322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.705563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe23c5f-9397-4f1c-a51b-d9823be1480c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.716281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.723117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 547/1092] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 11:04:17.729148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.740654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.741300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.741574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.741841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.753206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:17.758010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:17.759037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:17.760488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 546/1093] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:17.764926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.827722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06712-c15b-4fe0-8685-b7d17086de82 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.856566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.863014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.863597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95277f-374d-47ed-ab58-d288fff61838 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:17.863711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.863913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.864205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.865323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] RBAC: Authorizing `identity:get_project(project_id=d15133acb06044b1b028ebc5e29ccb9e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:17.866755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:17.868416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 548/1094] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:17.876037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.902066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:17.902666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:17.903076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:17.903243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:17.903896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:17.904225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:17.905569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:17.913314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 547/1095] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:17.919591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:17.938629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:17.977456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49141c50-eda5-4b60-92dd-3146ebab7075 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:18.014004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:18.014530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:18.014804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:18.015072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:18.015788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=d15133acb06044b1b028ebc5e29ccb9e, user.id=ec876f0814da42b8a9ee1a0ffdaf964d) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:18.016094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:18.017581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:18.023376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 549/1096] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/role_assignments?user.id=ec876f0814da42b8a9ee1a0ffdaf964d&scope.project.id=d15133acb06044b1b028ebc5e29ccb9e&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:18.024706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8769cb-3d7f-43ab-9b51-3a9dd60cae72 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:18.064248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:18.064874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:18.065152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:18.065424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:18.079381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:18.081357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:18.102328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:18.103553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:18.104545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:18.104545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:18.104545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:18.104545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 548/1097] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:18.110368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:18.220382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a1b27-ec60-4106-b331-0e8371276fe3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:18.247158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:18.255735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:18.256373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:18.256650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:18.256919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:18.257552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:18.257848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:18.259210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:18.271220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 550/1098] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:18.277161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:18.290494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:18.396701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90824b7-a45c-4644-bcee-f6e24f27646a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:18.438685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75436636-2955-42e6-a097-1af370873036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:18.439326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436636-2955-42e6-a097-1af370873036 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:18.439611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436636-2955-42e6-a097-1af370873036 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:18.440043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436636-2955-42e6-a097-1af370873036 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:18.449258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75436636-2955-42e6-a097-1af370873036 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:18.451856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75436636-2955-42e6-a097-1af370873036 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-75436636-2955-42e6-a097-1af370873036 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:04:18.464958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:18.466789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 11:04:18.467851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:18.472347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 549/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Aug 18 11:04:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:18.473783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:18.578505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b173f6-fa3b-4c55-bd39-98669f82d7e9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:18.612438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:18.613041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:18.613317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:18.613586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:18.614311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:18.614615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:18.616042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:18.635339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 551/1100] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Aug 18 11:04:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:18.641764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75436636-2955-42e6-a097-1af370873036 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:18.719498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:18.757535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75436636-2955-42e6-a097-1af370873036 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:18.795071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:18.795676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:18.795952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:18.796246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:18.868740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:18.870691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:18.942925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:18.980114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 550/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:18.986195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:19.039882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:19.095100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e2646-bbef-442f-959f-8b08973f589c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:19.132426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:19.132951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:19.133227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:19.133493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:19.144483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:19.145963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:19.162892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:19.164278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:04:19.165190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:19.165190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:04:19.165190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:19.165190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 552/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:19.171118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:19.277673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16284-8918-4871-8eba-fa055d5f23ce None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:19.313408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:19.314008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:19.314286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:19.314555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:19.315198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:19.315503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:19.316899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:19.328577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 551/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:19.334484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:19.430792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:19.435452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d02d9-43f7-4335-bc22-10c9bcf56da1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:19.468676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:19.469292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:19.469568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:19.469835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:19.476465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:19.477949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:04:19.489600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:19.490657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:19.491604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 553/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:04:19.497167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:19.525739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:19.597757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bda17-bbe2-4360-82fb-46a3e8bccf30 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:19.631384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:19.631982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:19.632289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:19.632563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:19.633253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:19.633555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:19.634973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6b64c-1828-47d2-9643-d9d18e8ff771 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:19.647337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 552/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:19.653281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:19.752238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b436aa-9fd7-45d1-8863-0d357201de5c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:19.785020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:19.785615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:19.785893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:19.786160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:19.786876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] RBAC: Adding query filter params (role.id=6cfa4326e893449aa1e4daa24e4223e7, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=1f43ff51c5f84d34978c46bce6db67ee) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:19.787178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:19.788627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338d4a7-e581-40d5-8c31-9b754ab4c01d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:19.794385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 554/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:19 2022] GET /identity/v3/role_assignments?user.id=1f43ff51c5f84d34978c46bce6db67ee&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=6cfa4326e893449aa1e4daa24e4223e7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:20.183582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6275a-6b92-4d9e-813f-b19236239c97 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:20.183868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6275a-6b92-4d9e-813f-b19236239c97 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:20.184239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6275a-6b92-4d9e-813f-b19236239c97 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:20.185704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 553/1107] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:20.190977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:20.191268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:20.191539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:20.210807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:20.212019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:20.229765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:20.286831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 555/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:20.557619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6275a-6b92-4d9e-813f-b19236239c97 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:20.656098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:20.669301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6275a-6b92-4d9e-813f-b19236239c97 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:20.703897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9994-21dd-4c70-9db6-c36f84616630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:20.704558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9994-21dd-4c70-9db6-c36f84616630 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:20.704835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9994-21dd-4c70-9db6-c36f84616630 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:20.705115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9994-21dd-4c70-9db6-c36f84616630 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:20.706223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c9994-21dd-4c70-9db6-c36f84616630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:20.707668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c9994-21dd-4c70-9db6-c36f84616630 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:20.709257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 554/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:20.715203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:20.813438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b278ece-a000-4acf-a0c3-fed5de60b0be None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:20.845776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911f2d9-c2dc-4568-a0b4-b738938578a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:20.846371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f2d9-c2dc-4568-a0b4-b738938578a6 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:20.846650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f2d9-c2dc-4568-a0b4-b738938578a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:20.846923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f2d9-c2dc-4568-a0b4-b738938578a6 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:20.847561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911f2d9-c2dc-4568-a0b4-b738938578a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:20.849013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911f2d9-c2dc-4568-a0b4-b738938578a6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:20.869936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 556/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:04:20 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:04:22.801409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430db6af-940f-40ca-b447-d370b4a7cf27 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:22.802021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430db6af-940f-40ca-b447-d370b4a7cf27 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:22.802021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430db6af-940f-40ca-b447-d370b4a7cf27 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:22.803283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 555/1111] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:22.808386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:22.808871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:22.808984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:22.826475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:22.827655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:22.829608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:22.894061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 557/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:23.177394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430db6af-940f-40ca-b447-d370b4a7cf27 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:23.279457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430db6af-940f-40ca-b447-d370b4a7cf27 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:23.317226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:23.317849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:23.318131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:23.318405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] PATH_INFO: `/v3/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:23.319808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] RBAC: Authorizing `identity:get_role(role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:23.321205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:23.322532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 556/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:23 2022] GET /identity/v3/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:23.329207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:23.425242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347c6d1-ada9-4d98-a6bf-ffb9c0e80737 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:23.458280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:23.458838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:23.459114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:23.459381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] PATH_INFO: `/v3/users/5275a80d4d10423283fcfeea400284ac` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:23.466461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:23.472759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] RBAC: Authorizing `identity:get_user(user_id=5275a80d4d10423283fcfeea400284ac)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:23.474187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:23.475774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 558/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:23 2022] GET /identity/v3/users/5275a80d4d10423283fcfeea400284ac => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:23.481642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:23.506973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:23.581852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddedc8-a8c5-465e-9f76-781cf4f76e88 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:23.613997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:23.614589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:23.614863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:23.615132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:23.679852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:23.681312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:23.704729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:23.778167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 557/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:23.784129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:23.881014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34263c3d-7872-4d2f-82f6-7c64d48e71ec None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:23.917597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf01d-388b-44bb-b36d-4e38c46bc051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:23.918793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf01d-388b-44bb-b36d-4e38c46bc051 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:23.919087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf01d-388b-44bb-b36d-4e38c46bc051 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:23.919353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf01d-388b-44bb-b36d-4e38c46bc051 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:23.920478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf01d-388b-44bb-b36d-4e38c46bc051 None admin] RBAC: Authorizing `identity:get_project(project_id=d15133acb06044b1b028ebc5e29ccb9e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:23.921891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf01d-388b-44bb-b36d-4e38c46bc051 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:23.923493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 559/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:23 2022] GET /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:23.929863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:24.025674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cd4ca-03ee-4528-8951-476a6fa02c08 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:24.049151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:24.060592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:24.061183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:24.061461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:24.061733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:24.062443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] RBAC: Adding query filter params (role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=d15133acb06044b1b028ebc5e29ccb9e, user.id=5275a80d4d10423283fcfeea400284ac) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:24.062750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:24.064175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7158ee1-ff11-4afc-9b6e-8c475d7d0133 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:24.069437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 558/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:23 2022] GET /identity/v3/role_assignments?user.id=5275a80d4d10423283fcfeea400284ac&scope.project.id=d15133acb06044b1b028ebc5e29ccb9e&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:25.959155 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf88fba-1b47-4995-b90a-b41f88f903d3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:25.960988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf88fba-1b47-4995-b90a-b41f88f903d3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:25.960988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf88fba-1b47-4995-b90a-b41f88f903d3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:25.961378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 560/1118] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:25.966652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:25.967114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:25.967215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:25.985444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:25.986604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:26.051827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 559/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:26.317426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf88fba-1b47-4995-b90a-b41f88f903d3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:26.379488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:26.417250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf88fba-1b47-4995-b90a-b41f88f903d3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:26.449850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:26.450456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:26.450731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:26.450999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] PATH_INFO: `/v3/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:26.452391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] RBAC: Authorizing `identity:get_role(role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:26.453753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:26.455058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 561/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:26 2022] GET /identity/v3/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:26.460929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:26.557408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0c6a6-f9fe-4254-82df-e737e907148e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:26.574353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:26.592010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:26.592646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:26.592922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:26.593189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] PATH_INFO: `/v3/users/5275a80d4d10423283fcfeea400284ac` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:26.594365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] RBAC: Authorizing `identity:get_user(user_id=5275a80d4d10423283fcfeea400284ac)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:26.595823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:26.597399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 560/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:26 2022] GET /identity/v3/users/5275a80d4d10423283fcfeea400284ac => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:26.603160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:26.700311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b4512-09b7-444b-b41a-6b302e9418c1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:26.728473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:26.733601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096285dd-a3ae-4c0b-8fb6-bd2a00568050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:26.734211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096285dd-a3ae-4c0b-8fb6-bd2a00568050 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:26.734489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096285dd-a3ae-4c0b-8fb6-bd2a00568050 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:26.734761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096285dd-a3ae-4c0b-8fb6-bd2a00568050 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:26.735867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096285dd-a3ae-4c0b-8fb6-bd2a00568050 None admin] RBAC: Authorizing `identity:get_project(project_id=d15133acb06044b1b028ebc5e29ccb9e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:26.737317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096285dd-a3ae-4c0b-8fb6-bd2a00568050 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:26.738914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 562/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:26 2022] GET /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:26.744803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:26.841830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c5f57-eb9b-47a0-878b-14e76bd9761f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:26.874531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bd3fb-2c96-4034-8f75-8d76b7b898f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:26.875158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bd3fb-2c96-4034-8f75-8d76b7b898f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:26.875429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bd3fb-2c96-4034-8f75-8d76b7b898f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:26.875696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bd3fb-2c96-4034-8f75-8d76b7b898f4 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e/users/5275a80d4d10423283fcfeea400284ac/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:26.877806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bd3fb-2c96-4034-8f75-8d76b7b898f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d15133acb06044b1b028ebc5e29ccb9e, user_id=5275a80d4d10423283fcfeea400284ac, role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:26.879219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bd3fb-2c96-4034-8f75-8d76b7b898f4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:26.891962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 561/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:04:26 2022] PUT /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e/users/5275a80d4d10423283fcfeea400284ac/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:04:28.770937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac9b2-cead-4845-a20c-40b7bffe070b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:28.771453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac9b2-cead-4845-a20c-40b7bffe070b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:28.771453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac9b2-cead-4845-a20c-40b7bffe070b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:28.772846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 563/1124] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:28.778815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:28.779250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:28.779357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:28.797520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:28.798695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:28.838961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:28.865047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 562/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:29.140081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac9b2-cead-4845-a20c-40b7bffe070b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:29.237787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac9b2-cead-4845-a20c-40b7bffe070b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:29.270493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:29.271092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:29.271363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:29.271627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] PATH_INFO: `/v3/roles/f229bfbb0384406e8c2b5c43d2f8b97c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:29.273004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] RBAC: Authorizing `identity:get_role(role_id=f229bfbb0384406e8c2b5c43d2f8b97c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:29.274345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:29.275618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 564/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:29 2022] GET /identity/v3/roles/f229bfbb0384406e8c2b5c43d2f8b97c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:29.282075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:29.383591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a22f3-4dc5-4eee-ac40-0658d27fb4de None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:29.416853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:29.417471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:29.417753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:29.418026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] PATH_INFO: `/v3/users/5275a80d4d10423283fcfeea400284ac` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:29.419229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=5275a80d4d10423283fcfeea400284ac)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:29.420697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:29.422261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 563/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:29 2022] GET /identity/v3/users/5275a80d4d10423283fcfeea400284ac => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:29.428081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:29.492927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:29.526477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ce9ac-df17-48b5-80c0-0d3a09e03cfd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:29.559787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:29.560361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:29.560644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:29.560916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:29.625758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:29.627203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:29.689369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:29.722582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 565/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:29.728506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:29.773695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:29.827192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda77c5-c99c-49d7-a573-9d6936a34bc6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:29.859623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd3dc5-b939-4e4e-acd3-5cc0b84ef0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:29.860257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3dc5-b939-4e4e-acd3-5cc0b84ef0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:29.860536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3dc5-b939-4e4e-acd3-5cc0b84ef0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:29.860806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3dc5-b939-4e4e-acd3-5cc0b84ef0a1 None admin] PATH_INFO: `/v3/projects/d15133acb06044b1b028ebc5e29ccb9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:29.861896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd3dc5-b939-4e4e-acd3-5cc0b84ef0a1 None admin] RBAC: Authorizing `identity:get_project(project_id=d15133acb06044b1b028ebc5e29ccb9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:29.863316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd3dc5-b939-4e4e-acd3-5cc0b84ef0a1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:29.864941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 564/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:29 2022] GET /identity/v3/projects/d15133acb06044b1b028ebc5e29ccb9e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:29.871030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:29.970286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7f797-500f-4ec1-b225-606b6ce81412 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:30.004426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:30.005022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:30.005300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:30.005572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:30.006290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] RBAC: Adding query filter params (role.id=f229bfbb0384406e8c2b5c43d2f8b97c, scope.project.id=d15133acb06044b1b028ebc5e29ccb9e, user.id=5275a80d4d10423283fcfeea400284ac) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:30.006596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:30.008014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8752f5-44e3-4cf1-81a9-053afdd328e4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:30.013899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 566/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:29 2022] GET /identity/v3/role_assignments?user.id=5275a80d4d10423283fcfeea400284ac&scope.project.id=d15133acb06044b1b028ebc5e29ccb9e&role.id=f229bfbb0384406e8c2b5c43d2f8b97c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:31.909379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2e8b7-460e-4f42-a0ac-dd5a68d87b1b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:31.909918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2e8b7-460e-4f42-a0ac-dd5a68d87b1b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:31.909918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2e8b7-460e-4f42-a0ac-dd5a68d87b1b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:31.911266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 565/1131] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:31.916297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:31.916722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:31.916829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:31.935111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:31.936301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:31.967252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:32.010184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 567/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:32.280635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2e8b7-460e-4f42-a0ac-dd5a68d87b1b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:32.379330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2e8b7-460e-4f42-a0ac-dd5a68d87b1b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:32.392513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:32.412764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44e9d9-2ebd-4d55-8838-d207078db99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:32.413374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44e9d9-2ebd-4d55-8838-d207078db99e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:32.413652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44e9d9-2ebd-4d55-8838-d207078db99e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:32.413921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44e9d9-2ebd-4d55-8838-d207078db99e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:32.415024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44e9d9-2ebd-4d55-8838-d207078db99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:32.416470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44e9d9-2ebd-4d55-8838-d207078db99e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:32.418011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 566/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:32.423959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:32.522400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9653bd-8e69-4ab6-afb6-2ff619d7fe3e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:32.554952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef43095-018c-4789-8ab7-953ba90bd97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:32.555492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef43095-018c-4789-8ab7-953ba90bd97b None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:32.555772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef43095-018c-4789-8ab7-953ba90bd97b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:32.556044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef43095-018c-4789-8ab7-953ba90bd97b None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:32.556712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef43095-018c-4789-8ab7-953ba90bd97b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:32.558128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef43095-018c-4789-8ab7-953ba90bd97b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:32.585831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 568/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:04:32 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:34.499870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd4342-3d61-48a4-b6f9-d734bb11e4d6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:34.500450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd4342-3d61-48a4-b6f9-d734bb11e4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:34.500450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd4342-3d61-48a4-b6f9-d734bb11e4d6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:34.501788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 567/1135] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:34.507015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:34.507394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:34.507560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:34.526514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:34.528313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:34.537228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:34.594000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 569/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:34.859370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd4342-3d61-48a4-b6f9-d734bb11e4d6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:34.960463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd4342-3d61-48a4-b6f9-d734bb11e4d6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:34.993488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13737a1-d9b3-4ff2-bfaf-37e51843f6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:34.994094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13737a1-d9b3-4ff2-bfaf-37e51843f6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:34.994372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13737a1-d9b3-4ff2-bfaf-37e51843f6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:34.994646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13737a1-d9b3-4ff2-bfaf-37e51843f6f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:34.995753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13737a1-d9b3-4ff2-bfaf-37e51843f6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:34.997907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13737a1-d9b3-4ff2-bfaf-37e51843f6f1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:34.999740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 568/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:04:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:35.005577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:35.100147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ef6c2-2825-4eea-8467-c1c66851c7ce None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:35.133501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48147e9-f300-4632-bf29-d392f456880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:35.134037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48147e9-f300-4632-bf29-d392f456880d None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:35.134309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48147e9-f300-4632-bf29-d392f456880d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:35.134572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48147e9-f300-4632-bf29-d392f456880d None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:35.135203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48147e9-f300-4632-bf29-d392f456880d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:35.136644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48147e9-f300-4632-bf29-d392f456880d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:35.152346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:35.157613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 570/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:04:35 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:37.091837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320da72-f841-422d-9857-aaa9675cd857 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:37.092381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320da72-f841-422d-9857-aaa9675cd857 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:37.092473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320da72-f841-422d-9857-aaa9675cd857 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:37.093731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 569/1139] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:37.098775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:37.099205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:37.099321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:37.117081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:37.118267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:37.182954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 571/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:37.447902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320da72-f841-422d-9857-aaa9675cd857 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:37.498692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:37.548108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320da72-f841-422d-9857-aaa9675cd857 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:37.581473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:37.582089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:37.582368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:37.582641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:37.593174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:37.597867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:37.599041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:37.599983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 570/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:37.605175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:37.706202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0710be-54df-44b6-add3-aba459c448f6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:37.734546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:37.740767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:37.741366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:37.741647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:37.741918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:37.742572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:37.742875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:37.744250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:37.752362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 572/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:04:37.758123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:37.857153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b8989-bbf8-461d-9628-6440504c8c25 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:37.866080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:37.890765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:37.891372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:37.891650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:37.891918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] PATH_INFO: `/v3/users/dbfeb2b29bb14806b4c977553ddc6de0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:37.905121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] RBAC: Authorizing `identity:get_user(user_id=dbfeb2b29bb14806b4c977553ddc6de0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:37.906544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:37.908116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 571/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:37 2022] GET /identity/v3/users/dbfeb2b29bb14806b4c977553ddc6de0 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:37.913971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:38.019135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0590e-713e-4900-9f4c-33f6f1ad351b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:38.052785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:38.053386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:38.053665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:38.053938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:38.110579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:38.120061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:38.121828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:38.216427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 573/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:38.222451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:38.319194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca39d8-c0aa-4286-aed7-6db92979f2f1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:38.357997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a003dc-4993-463b-88d8-a9c998c3fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:38.358615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a003dc-4993-463b-88d8-a9c998c3fe94 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:38.358901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a003dc-4993-463b-88d8-a9c998c3fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:38.359173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a003dc-4993-463b-88d8-a9c998c3fe94 None admin] PATH_INFO: `/v3/projects/1314f58532a54b52958ce4516779f4c0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:38.360305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a003dc-4993-463b-88d8-a9c998c3fe94 None admin] RBAC: Authorizing `identity:get_project(project_id=1314f58532a54b52958ce4516779f4c0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:38.361734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a003dc-4993-463b-88d8-a9c998c3fe94 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:38.363350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 572/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:38 2022] GET /identity/v3/projects/1314f58532a54b52958ce4516779f4c0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:38.369631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:38.460259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:38.469354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33643-0a86-4cfe-8a79-de8c70e2c0ca None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:38.503383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:38.503976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:38.504284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:38.504559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:38.505273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=1314f58532a54b52958ce4516779f4c0, user.id=dbfeb2b29bb14806b4c977553ddc6de0) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:38.505580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:38.506997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50404-9373-4daa-8322-ed537c6d65a8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:38.513071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 574/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:38 2022] GET /identity/v3/role_assignments?user.id=dbfeb2b29bb14806b4c977553ddc6de0&scope.project.id=1314f58532a54b52958ce4516779f4c0&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:40.373799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864aade-e11a-442e-a28e-abbaeb4b0b78 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:40.374310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864aade-e11a-442e-a28e-abbaeb4b0b78 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:40.374310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864aade-e11a-442e-a28e-abbaeb4b0b78 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:40.375609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 573/1147] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:40.380680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:40.381083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:40.381223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:40.398903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:40.400092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:40.464057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 575/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:04:40.737593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864aade-e11a-442e-a28e-abbaeb4b0b78 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:40.771986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:40.837419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864aade-e11a-442e-a28e-abbaeb4b0b78 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:40.870716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:40.871377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:40.871653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:40.871924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:40.882026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:40.886649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:40.887826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:40.889104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 574/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:40.893752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:40.976246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:40.991326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e18f4-1d69-43be-b046-c0e1e1dd0e0a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:41.025278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:41.025912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:41.026191 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:41.026460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:41.027115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:41.028048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:41.029758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:41.037426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 576/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:41.043449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:41.134750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:41.142255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb51e6-ec1c-487e-b463-e6845e15b3c8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:41.174960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:41.175566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:41.175843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:41.176114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] PATH_INFO: `/v3/users/dbfeb2b29bb14806b4c977553ddc6de0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:41.177244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] RBAC: Authorizing `identity:get_user(user_id=dbfeb2b29bb14806b4c977553ddc6de0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:41.178663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:41.180231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 575/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:41 2022] GET /identity/v3/users/dbfeb2b29bb14806b4c977553ddc6de0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:41.186019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:41.283596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81a3c-dda7-4537-8de9-08607be8b8b8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:41.316151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef16d-3fa4-49cf-9dbb-9c20d1f7f1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:41.316903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef16d-3fa4-49cf-9dbb-9c20d1f7f1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:41.317169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef16d-3fa4-49cf-9dbb-9c20d1f7f1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:41.317435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef16d-3fa4-49cf-9dbb-9c20d1f7f1e8 None admin] PATH_INFO: `/v3/projects/1314f58532a54b52958ce4516779f4c0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:41.318542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ef16d-3fa4-49cf-9dbb-9c20d1f7f1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=1314f58532a54b52958ce4516779f4c0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:41.320081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ef16d-3fa4-49cf-9dbb-9c20d1f7f1e8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:41.321837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 577/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:41 2022] GET /identity/v3/projects/1314f58532a54b52958ce4516779f4c0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:41.329013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:41.427100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a2eb-eb09-4f6f-8753-1c91d4d1931b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:41.460242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc050e-a240-482f-b0d1-9295f95ef0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:41.460885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc050e-a240-482f-b0d1-9295f95ef0ae None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:41.461161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc050e-a240-482f-b0d1-9295f95ef0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:41.461431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc050e-a240-482f-b0d1-9295f95ef0ae None admin] PATH_INFO: `/v3/projects/1314f58532a54b52958ce4516779f4c0/users/dbfeb2b29bb14806b4c977553ddc6de0/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:41.463307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc050e-a240-482f-b0d1-9295f95ef0ae None admin] RBAC: Authorizing `identity:create_grant(project_id=1314f58532a54b52958ce4516779f4c0, user_id=dbfeb2b29bb14806b4c977553ddc6de0, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:41.464766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc050e-a240-482f-b0d1-9295f95ef0ae None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:41.477461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 576/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:04:41 2022] PUT /identity/v3/projects/1314f58532a54b52958ce4516779f4c0/users/dbfeb2b29bb14806b4c977553ddc6de0/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:04:43.352483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc48f25-208d-486a-a9f5-872be976c842 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:43.353012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc48f25-208d-486a-a9f5-872be976c842 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:43.353012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc48f25-208d-486a-a9f5-872be976c842 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:43.354300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 578/1154] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:43.359654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:43.360061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:43.360339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:43.378231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:43.379400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:43.395803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:43.445476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 577/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:43.712905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc48f25-208d-486a-a9f5-872be976c842 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:43.731260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:43.812670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc48f25-208d-486a-a9f5-872be976c842 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:43.844932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:43.845537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:43.845814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:43.846085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:43.856424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:43.861027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:43.861941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:43.862788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 579/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:43.868285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:43.966623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e257c2-db4c-41b2-acd3-652289f8770f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:44.001011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:44.001612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:44.001894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:44.002167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:44.002833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:44.003136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:44.004530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:44.013040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 578/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:44.019576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:44.095526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:44.118758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2b748-7af6-46f0-b4ef-f3c85f8f92dc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:44.151576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:44.152206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:44.152520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:44.152795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] PATH_INFO: `/v3/users/dbfeb2b29bb14806b4c977553ddc6de0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:44.153905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] RBAC: Authorizing `identity:get_user(user_id=dbfeb2b29bb14806b4c977553ddc6de0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:44.155339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:44.156940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 580/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:44 2022] GET /identity/v3/users/dbfeb2b29bb14806b4c977553ddc6de0 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:44.162954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:44.179522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:44.261768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacc0c-52fc-458b-926e-aa0cf1160411 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:44.294527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:44.295118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:44.295397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:44.295664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:44.362584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:44.364028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:44.378364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:44.460730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 579/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:44.467502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:44.564728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b2ec0-506a-41c1-9f31-198c2d959e06 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:44.597140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1ad1-c817-4e6f-952b-66338e92ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:44.597744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1ad1-c817-4e6f-952b-66338e92ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:44.598025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1ad1-c817-4e6f-952b-66338e92ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:44.598298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1ad1-c817-4e6f-952b-66338e92ccf5 None admin] PATH_INFO: `/v3/projects/1314f58532a54b52958ce4516779f4c0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:44.599398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c1ad1-c817-4e6f-952b-66338e92ccf5 None admin] RBAC: Authorizing `identity:get_project(project_id=1314f58532a54b52958ce4516779f4c0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:44.600885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c1ad1-c817-4e6f-952b-66338e92ccf5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:44.602490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 581/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:44 2022] GET /identity/v3/projects/1314f58532a54b52958ce4516779f4c0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:44.609355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:44.711597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6081ae-9de9-4be9-bb82-8bca01ae5b5b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:44.726999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:44.748606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:44.749204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:44.749483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:44.749755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:44.750473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=1314f58532a54b52958ce4516779f4c0, user.id=dbfeb2b29bb14806b4c977553ddc6de0) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:44.750781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:44.752230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88fd94-b0df-4916-a65e-419a19dd27d6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:44.757958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 580/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:44 2022] GET /identity/v3/role_assignments?user.id=dbfeb2b29bb14806b4c977553ddc6de0&scope.project.id=1314f58532a54b52958ce4516779f4c0&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:46.643568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3c132-021a-4e55-93af-b4c0188f783a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:46.644136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3c132-021a-4e55-93af-b4c0188f783a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:46.644136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3c132-021a-4e55-93af-b4c0188f783a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:46.645452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 582/1162] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:46.650496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:46.650925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:46.651043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:46.669793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:46.670967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:46.736369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 581/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:47.005255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3c132-021a-4e55-93af-b4c0188f783a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:47.020945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:47.105236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3c132-021a-4e55-93af-b4c0188f783a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:47.137477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:47.138053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:47.138325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:47.138588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:47.203200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:47.204661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:47.218086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:47.300145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 583/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:47.305967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:47.403662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d6df-3039-4159-81bc-d306a0bd1888 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:47.412599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:47.437035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:47.437643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:47.437914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:47.438181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:47.444585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:47.446004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a5dc474c-4b41-4cc0-9b5e-dbc53c8e02a1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:47.456919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:47.458115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:04:47.459002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 582/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:04:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:47.464012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:47.560939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7382a1-98e2-4dad-bb5e-b98e2580e450 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:47.573519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:47.594542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:47.595134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:47.595410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:47.595681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:47.596346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:47.596648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:47.597997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab1d44-710f-46a9-bff3-ece05f941e55 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:47.604856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 584/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Aug 18 11:04:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:49.497100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2e2a-cc3a-4d13-a0fe-de8af68c2479 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:49.497633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2e2a-cc3a-4d13-a0fe-de8af68c2479 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:49.497633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2e2a-cc3a-4d13-a0fe-de8af68c2479 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:49.499229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 583/1167] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:49.504439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7624a3-1e68-451d-9693-a39c6d6e6732 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:49.504844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7624a3-1e68-451d-9693-a39c6d6e6732 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:49.504992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7624a3-1e68-451d-9693-a39c6d6e6732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:49.522530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9e7624a3-1e68-451d-9693-a39c6d6e6732 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:49.523816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9e7624a3-1e68-451d-9693-a39c6d6e6732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:49.588587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 585/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:49.862966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2e2a-cc3a-4d13-a0fe-de8af68c2479 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:49.965845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2e2a-cc3a-4d13-a0fe-de8af68c2479 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:50.002154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0ae7d-745e-4cff-a8ea-87d79e976a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:50.002764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ae7d-745e-4cff-a8ea-87d79e976a95 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:50.003039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ae7d-745e-4cff-a8ea-87d79e976a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:50.003316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ae7d-745e-4cff-a8ea-87d79e976a95 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:50.003987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0ae7d-745e-4cff-a8ea-87d79e976a95 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:50.005412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0ae7d-745e-4cff-a8ea-87d79e976a95 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:50.022193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 584/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Aug 18 11:04:49 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:04:51.916661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f4275-9c22-4eeb-9ca6-092e5d2d8c4d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:51.917225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f4275-9c22-4eeb-9ca6-092e5d2d8c4d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:51.917225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f4275-9c22-4eeb-9ca6-092e5d2d8c4d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:51.918624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 586/1170] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:51.923835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002394bd-f047-4769-a291-e053f87597fa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:51.924264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002394bd-f047-4769-a291-e053f87597fa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:51.924415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002394bd-f047-4769-a291-e053f87597fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:51.942624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-002394bd-f047-4769-a291-e053f87597fa None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:51.943812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-002394bd-f047-4769-a291-e053f87597fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:51.967560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:52.012467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 585/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:52.281783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f4275-9c22-4eeb-9ca6-092e5d2d8c4d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:52.379691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f4275-9c22-4eeb-9ca6-092e5d2d8c4d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:52.387201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:52.413974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e8873-176a-4a74-a480-83617857b8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:52.414585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e8873-176a-4a74-a480-83617857b8db None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:52.414863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e8873-176a-4a74-a480-83617857b8db None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:52.415136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e8873-176a-4a74-a480-83617857b8db None admin] PATH_INFO: `/v3/domains/181f481b9a62407fac074e4c24762ba2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:52.425403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e8873-176a-4a74-a480-83617857b8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=181f481b9a62407fac074e4c24762ba2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:52.426842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e8873-176a-4a74-a480-83617857b8db None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:52.428444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 587/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Aug 18 11:04:52 2022] GET /identity/v3/domains/181f481b9a62407fac074e4c24762ba2 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:52.434363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002394bd-f047-4769-a291-e053f87597fa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:52.532146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002394bd-f047-4769-a291-e053f87597fa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:52.564566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aae212-2dd0-47da-8263-302368d4b080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:52.565150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aae212-2dd0-47da-8263-302368d4b080 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:52.565428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aae212-2dd0-47da-8263-302368d4b080 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:52.565700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aae212-2dd0-47da-8263-302368d4b080 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:52.566337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aae212-2dd0-47da-8263-302368d4b080 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:52.567755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aae212-2dd0-47da-8263-302368d4b080 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:52.593307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 586/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:04:52 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:04:54.519712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f51a73-fb41-4949-b145-a76e5292dd60 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:54.520274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f51a73-fb41-4949-b145-a76e5292dd60 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:54.520274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f51a73-fb41-4949-b145-a76e5292dd60 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:54.521648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 588/1174] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:54.526935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:54.527978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:54.528063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:54.545846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:54.547024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:54.549756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:54.613389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 587/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:54.884621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f51a73-fb41-4949-b145-a76e5292dd60 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:54.983595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f51a73-fb41-4949-b145-a76e5292dd60 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:55.017253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27539a4-2268-44bd-8133-e01d3cf94b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:55.017867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27539a4-2268-44bd-8133-e01d3cf94b8a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:55.018142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27539a4-2268-44bd-8133-e01d3cf94b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:55.018410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27539a4-2268-44bd-8133-e01d3cf94b8a None admin] PATH_INFO: `/v3/domains/181f481b9a62407fac074e4c24762ba2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:55.019517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27539a4-2268-44bd-8133-e01d3cf94b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=181f481b9a62407fac074e4c24762ba2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:55.020960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27539a4-2268-44bd-8133-e01d3cf94b8a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:55.022495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 589/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Aug 18 11:04:54 2022] GET /identity/v3/domains/181f481b9a62407fac074e4c24762ba2 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:55.028915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:55.128209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d866b-cce0-42db-8485-2064c0c19d17 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:55.161992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afcbb1-8d0a-4e4e-9fc9-e58bd3c89895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:55.162591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afcbb1-8d0a-4e4e-9fc9-e58bd3c89895 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:55.162870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afcbb1-8d0a-4e4e-9fc9-e58bd3c89895 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:55.163143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afcbb1-8d0a-4e4e-9fc9-e58bd3c89895 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:55.163789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afcbb1-8d0a-4e4e-9fc9-e58bd3c89895 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:55.165247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afcbb1-8d0a-4e4e-9fc9-e58bd3c89895 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:55.175839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:55.187913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 588/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:04:55 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:04:57.120009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177075c-4680-403a-ab56-1d3b3f2c7910 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:57.120590 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177075c-4680-403a-ab56-1d3b3f2c7910 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:57.120590 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177075c-4680-403a-ab56-1d3b3f2c7910 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:57.122610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 590/1178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:04:57.127940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:57.128358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:57.128521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:57.146707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:04:57.147892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:04:57.212195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 589/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:04:57.481069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177075c-4680-403a-ab56-1d3b3f2c7910 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:57.516043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:57.580973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177075c-4680-403a-ab56-1d3b3f2c7910 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:57.612780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:57.613382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:57.613655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:57.613921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:57.624896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:04:57.629426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:04:57.630606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:04:57.631731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 591/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:04:57.636544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:57.731864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64704f3e-8261-49bc-8d20-683774c129b5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:57.757177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:57.766072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:57.766664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:57.766938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:57.767205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:57.767859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:57.768164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:57.769537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:57.777388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 590/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:57.783252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:57.875771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:57.882751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba82cff-74da-4ea3-883f-ec5e316ac7e8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:57.915174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:57.915791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:57.916066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:57.916373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] PATH_INFO: `/v3/users/ce993fefb5c14dc29d11d2b4e508f239` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:57.929641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] RBAC: Authorizing `identity:get_user(user_id=ce993fefb5c14dc29d11d2b4e508f239)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:57.931068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:57.932674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 592/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:04:57 2022] GET /identity/v3/users/ce993fefb5c14dc29d11d2b4e508f239 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:57.938784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:58.038972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac096195-63e8-4cae-b78a-dc546de13865 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:58.071749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:58.072370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:58.072648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:58.072917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:58.128759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:58.141240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:58.142682 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:58.237333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 591/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:04:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:04:58.243330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:58.344889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130676b-39e6-4196-9f8e-13771807ca83 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:58.376912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5482bf7-e796-4ae4-b392-7e67119164de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:58.377521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5482bf7-e796-4ae4-b392-7e67119164de None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:58.377794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5482bf7-e796-4ae4-b392-7e67119164de None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:58.378060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5482bf7-e796-4ae4-b392-7e67119164de None admin] PATH_INFO: `/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:58.379141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5482bf7-e796-4ae4-b392-7e67119164de None admin] RBAC: Authorizing `identity:get_project(project_id=50f84d9f89a84031ba2d7f0f36aa188c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:58.380595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5482bf7-e796-4ae4-b392-7e67119164de None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:58.382192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 593/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:04:58 2022] GET /identity/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:04:58.389064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:04:58.473582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:04:58.486264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457137b-8d13-4ab1-969a-3f4fde1f02de None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:04:58.519535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:04:58.520124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:04:58.520434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:04:58.520708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:04:58.521427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=50f84d9f89a84031ba2d7f0f36aa188c, user.id=ce993fefb5c14dc29d11d2b4e508f239) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:04:58.521762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:04:58.523185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ee5c8-f5ae-4965-bac8-96f8ccd2e4be None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:04:58.529158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 592/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:04:58 2022] GET /identity/v3/role_assignments?user.id=ce993fefb5c14dc29d11d2b4e508f239&scope.project.id=50f84d9f89a84031ba2d7f0f36aa188c&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:00.383613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e401383-8ffd-4ba6-b02b-5bf7ac8d14fd None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:00.384138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e401383-8ffd-4ba6-b02b-5bf7ac8d14fd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:00.384138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e401383-8ffd-4ba6-b02b-5bf7ac8d14fd None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:00.385496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 594/1186] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:05:00.391281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:00.391683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:00.391825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:00.410374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:05:00.411580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:05:00.477020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 593/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:05:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:05:00.747221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e401383-8ffd-4ba6-b02b-5bf7ac8d14fd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:00.769357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:00.845976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e401383-8ffd-4ba6-b02b-5bf7ac8d14fd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:00.878478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:00.879085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:00.879356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:00.879620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:00.889855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:05:00.894420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:05:00.895358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:05:00.896253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 595/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:05:00.901512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:00.980023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:00.998816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b30999-4ee2-450f-81ce-fd7fdbff5f8a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:01.032171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:01.032791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:01.033070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:01.033343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:01.034004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:05:01.034306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:01.035672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:01.043612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 594/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:01.049421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:01.129679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:01.149093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984a74d-535c-4c47-b059-60046f9eff4b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:01.181266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:01.181883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:01.182172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:01.182433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] PATH_INFO: `/v3/users/ce993fefb5c14dc29d11d2b4e508f239` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:01.184035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] RBAC: Authorizing `identity:get_user(user_id=ce993fefb5c14dc29d11d2b4e508f239)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:01.185598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:01.187172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 596/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:05:01 2022] GET /identity/v3/users/ce993fefb5c14dc29d11d2b4e508f239 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:01.193160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:01.291268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab1e6-bd2d-491b-a2d7-fae6ef44beb3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:01.324593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ce4e-238e-40b2-8c14-d943e8b796d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:01.325200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ce4e-238e-40b2-8c14-d943e8b796d4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:01.325473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ce4e-238e-40b2-8c14-d943e8b796d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:01.325740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ce4e-238e-40b2-8c14-d943e8b796d4 None admin] PATH_INFO: `/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:01.326825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ce4e-238e-40b2-8c14-d943e8b796d4 None admin] RBAC: Authorizing `identity:get_project(project_id=50f84d9f89a84031ba2d7f0f36aa188c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:01.328272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ce4e-238e-40b2-8c14-d943e8b796d4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:01.329860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 595/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:05:01 2022] GET /identity/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:01.335679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:01.440629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb42d87-4ce3-43cc-89a0-4f34ae8253b8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:01.473352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f448fb-c9d4-4d98-b245-01cf2fc69aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:01.473991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f448fb-c9d4-4d98-b245-01cf2fc69aab None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:01.474266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f448fb-c9d4-4d98-b245-01cf2fc69aab None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:01.474531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f448fb-c9d4-4d98-b245-01cf2fc69aab None admin] PATH_INFO: `/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c/users/ce993fefb5c14dc29d11d2b4e508f239/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:01.476461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f448fb-c9d4-4d98-b245-01cf2fc69aab None admin] RBAC: Authorizing `identity:create_grant(project_id=50f84d9f89a84031ba2d7f0f36aa188c, user_id=ce993fefb5c14dc29d11d2b4e508f239, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:01.477896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f448fb-c9d4-4d98-b245-01cf2fc69aab None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:01.491034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 597/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:05:01 2022] PUT /identity/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c/users/ce993fefb5c14dc29d11d2b4e508f239/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:05:03.387850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1174d3f-7902-4eac-abd5-d5648e18ba2f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:03.388405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1174d3f-7902-4eac-abd5-d5648e18ba2f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:03.388405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1174d3f-7902-4eac-abd5-d5648e18ba2f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:03.389762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 596/1193] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:05:03.394992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:03.395448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:03.395543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:03.413058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:05:03.414239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:05:03.416166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:03.482647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 598/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:05:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:05:03.749832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1174d3f-7902-4eac-abd5-d5648e18ba2f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:03.765108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:03.851038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1174d3f-7902-4eac-abd5-d5648e18ba2f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:03.883738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:03.884380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:03.884658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:03.884926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:03.895208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:05:03.899806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:05:03.900951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:05:03.901940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 597/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:05:03.907564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:04.004017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d62206-fed8-40a4-8818-bce92bf52d25 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:04.037524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:04.038125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:04.038401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:04.038671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:04.039323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:05:04.039626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:04.041041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:04.046678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:04.052242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 599/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:04.057968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:04.131320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:04.161235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e5f4-e6b5-42e0-bb5e-e07229e75616 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:04.194952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:04.195568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:04.195845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:04.196116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] PATH_INFO: `/v3/users/ce993fefb5c14dc29d11d2b4e508f239` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:04.197253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] RBAC: Authorizing `identity:get_user(user_id=ce993fefb5c14dc29d11d2b4e508f239)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:04.198691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:04.201065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 598/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Aug 18 11:05:04 2022] GET /identity/v3/users/ce993fefb5c14dc29d11d2b4e508f239 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:04.206940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:04.304750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349e273-ddd8-487f-93bf-8b2db2aa004d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:04.337576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:04.338156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:04.338432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:04.338701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:04.401798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:04.405081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:04.406862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:04.501554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 600/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:05:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:05:04.507485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:04.605183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e32a3-74fe-4d0d-9a31-8f441240dfe4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:04.638437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c341d-0942-48f7-b890-8f8988b67268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:04.639049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c341d-0942-48f7-b890-8f8988b67268 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:04.639324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c341d-0942-48f7-b890-8f8988b67268 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:04.639590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c341d-0942-48f7-b890-8f8988b67268 None admin] PATH_INFO: `/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:04.640719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c341d-0942-48f7-b890-8f8988b67268 None admin] RBAC: Authorizing `identity:get_project(project_id=50f84d9f89a84031ba2d7f0f36aa188c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:04.642138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c341d-0942-48f7-b890-8f8988b67268 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:04.643737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 599/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Aug 18 11:05:04 2022] GET /identity/v3/projects/50f84d9f89a84031ba2d7f0f36aa188c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:05:04.649885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:05:04.743350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:04.746775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369de10-7651-49a8-a0bf-3ac6b205177c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:05:04.781596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:05:04.782183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:05:04.782461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:05:04.782732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:05:04.783445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=50f84d9f89a84031ba2d7f0f36aa188c, user.id=ce993fefb5c14dc29d11d2b4e508f239) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:05:04.783748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:05:04.785199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5832ee5-7ba7-4b00-8122-bd4cfc4fc5f7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:05:04.790786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 601/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:05:04 2022] GET /identity/v3/role_assignments?user.id=ce993fefb5c14dc29d11d2b4e508f239&scope.project.id=50f84d9f89a84031ba2d7f0f36aa188c&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:05:14.633706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:05:14.788260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=18 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:00.431007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d156b-b1ba-43cf-8ec7-db3d95af2356 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:00.431007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d156b-b1ba-43cf-8ec7-db3d95af2356 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:00.431007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d156b-b1ba-43cf-8ec7-db3d95af2356 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:00.432052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 600/1201] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:06:00.437659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:00.438130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:00.438218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:00.459773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:00.461216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:00.566624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 602/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:00.873295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d156b-b1ba-43cf-8ec7-db3d95af2356 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:01.010692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d156b-b1ba-43cf-8ec7-db3d95af2356 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:01.058028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:01.059015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:01.059490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:01.060040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:01.077552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:01.079335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:01.098304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:01.099723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:01.100817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:01.100817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:01.100817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:01.100817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 601/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:06:01.108581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:01.236667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a64b6-5d36-4e74-b662-d26ba1dfbb86 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:01.277934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:01.285872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:01.286511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:01.286898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:01.287177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:01.287830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:01.288136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:01.293511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda46420-1494-43cd-bff7-dcc955f500bb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:01.306891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 603/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:01.314789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:01.380942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:01.444858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0709da-079c-4577-a57e-c9db3fa38a0c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:01.488517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090ea66-23ec-47ca-80bf-68dc622d0e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:01.489214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090ea66-23ec-47ca-80bf-68dc622d0e05 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:01.489623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090ea66-23ec-47ca-80bf-68dc622d0e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:01.489911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090ea66-23ec-47ca-80bf-68dc622d0e05 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:01.490720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090ea66-23ec-47ca-80bf-68dc622d0e05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:01.492444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090ea66-23ec-47ca-80bf-68dc622d0e05 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:01.525756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 602/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Aug 18 11:06:01 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:06:03.736554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524d6e1-f6e8-4319-bb25-a956b8fa54e3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:03.737129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524d6e1-f6e8-4319-bb25-a956b8fa54e3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:03.737129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524d6e1-f6e8-4319-bb25-a956b8fa54e3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:03.738676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 604/1206] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:03.744378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:03.744801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:03.745126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:03.766908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:03.768147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:03.856230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 603/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:04.146823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524d6e1-f6e8-4319-bb25-a956b8fa54e3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:04.295018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524d6e1-f6e8-4319-bb25-a956b8fa54e3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:04.337821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:04.340608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:04.340608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:04.340608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:04.353850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:06:04.359656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:04.360665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:04.361526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 605/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:06:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:04.367547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:04.414457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:04.494174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f731bc-95b0-4780-b515-6a031a5f3ca8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:04.535400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:04.536005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:04.536440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:04.536845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:04.537516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:04.537819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:04.539194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:04.549285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 604/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:04.556323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:04.617103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:04.679910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753d9fe-3837-4026-b094-791c0bc45da1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:04.719732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:04.720385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:04.720664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:04.720933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:04.733004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:04.734705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:04.751043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:04.757522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:04.758618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:04.758618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:04.758618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:04.758618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 606/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:04.761281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:04.883341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:04.887963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d80376-ce48-4ad9-bb31-4612c9773eae None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:04.931654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:04.932297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:04.933180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:04.933490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:04.934129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:04.934428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:04.935789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:04.948060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 605/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:04.954649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:05.073077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05af1-2cbc-4f27-b6af-528c44f8ec88 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:05.075350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:05.110807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:05.111465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:05.111779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:05.112058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:05.121850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:05.123623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:06:05.140101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:05.141251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:05.142526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 607/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:05.147768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:05.270097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c0104-b5e4-4060-b99f-59629b41becf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:05.317616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:05.318237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:05.318608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:05.318948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:05.319671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:05.319981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:05.321470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:05.343168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 606/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Aug 18 11:06:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:05.349530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:05.471074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe846f02-7970-4729-996e-44e174e080c6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:05.515228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58975c52-b6be-4981-be50-e393f814368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:05.515872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58975c52-b6be-4981-be50-e393f814368e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:05.516122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58975c52-b6be-4981-be50-e393f814368e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:05.517497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58975c52-b6be-4981-be50-e393f814368e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:05.557195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:05.607203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58975c52-b6be-4981-be50-e393f814368e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:05.608784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58975c52-b6be-4981-be50-e393f814368e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:05.728606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 608/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:06:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:06:05.734326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:05.760065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:05.852738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f34da-ba7d-4e62-8b8c-3bdf5f5b8d9b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:05.894677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:05.895386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:05.895716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:05.896005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:05.908244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:05.909667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:05.926056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:05.927167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:05.933130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:05.933130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:05.933130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:05.933130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 607/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:05.933382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58975c52-b6be-4981-be50-e393f814368e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:05.997828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:06.038891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58975c52-b6be-4981-be50-e393f814368e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:06.078398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:06.079142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:06.079530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:06.079899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:06.080680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:06.081120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:06.082750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:06.101559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 609/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:06.103545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:06.187328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:06.220972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cacbd-a400-42c7-83f1-e341cb55ec88 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:06.257813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:06.258336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:06.258618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:06.258890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:06.267179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:06.268760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:06:06.281260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:06.282248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:06.288134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 608/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:06.290362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:06.411119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc05-f04a-43b7-b65f-74dad03c2534 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:06.442613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:06.456622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:06.457238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:06.457489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:06.457762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:06.458457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:06.458749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:06.460163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072da93-aa23-4598-913f-9894139a4748 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:06.474599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 610/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:06.483098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:06.607771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a859df-dcff-43ff-8273-50807b7ad35d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:06.640329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:06.647944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:06.648663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:06.649006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:06.649425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:06.650135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=f669319338ab4c6ea22dead596391789) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:06.650501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:06.652252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a1bd0-b0cb-4962-9fa2-264453260a89 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:06.658164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 609/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:06:06 2022] GET /identity/v3/role_assignments?user.id=f669319338ab4c6ea22dead596391789&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:08.617991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67de0ec-dc23-4b85-86b2-c37129c2e6c3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:08.618556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67de0ec-dc23-4b85-86b2-c37129c2e6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:08.618667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67de0ec-dc23-4b85-86b2-c37129c2e6c3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:08.620645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 611/1220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:06:08.626056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:08.626498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:08.626596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:08.644997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:08.646147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:08.719802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 610/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:06:08.985261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67de0ec-dc23-4b85-86b2-c37129c2e6c3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:09.092784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67de0ec-dc23-4b85-86b2-c37129c2e6c3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:09.129669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:09.130301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:09.130577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:09.130846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:09.141285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:06:09.145996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:09.146990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:09.147871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 612/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:09.155177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:09.271680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bfe79-1232-4da4-9aee-90800a2e3935 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:09.297700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:09.310564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:09.311293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:09.311590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:09.311912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:09.312761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:09.313038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:09.314721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:09.331887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 611/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:09.333662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:09.394361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:09.455897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14dbdd-1585-4745-8264-a54282ece9d9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:09.492090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:09.492788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:09.493068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:09.493339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:09.503772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:09.505392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:09.521130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:09.522322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:09.523254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:09.523254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:09.523254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:09.523254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 613/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:09.529438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:09.639860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de537a0-d39c-46cc-907e-3d45685a8df3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:09.676563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:09.677292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:09.677605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:09.677885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:09.678636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:09.678973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:09.680442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:09.694068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 612/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:09.700519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:09.811343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:09.818601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba299a4-1bd9-468e-be4f-b4dc4d7b2479 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:09.854092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:09.855328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:09.855605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:09.855874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:09.863453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:09.864918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:06:09.876382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:09.880294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:09.881536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 614/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:09.883387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:09.932953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:09.988468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4b447-2fb9-453c-bb86-678851c319e5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:10.030454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:10.031070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:10.031350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:10.031627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:10.032469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:10.032771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:10.034211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:10.053042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 613/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Aug 18 11:06:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:10.059005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:10.168413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f1040-5bef-44e8-8c9e-7cb0395853a3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:10.205610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:10.206299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:10.206575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:10.206838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:10.219292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:10.220820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:10.237286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:10.238493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:10.239611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:10.239611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:10.239611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:10.239611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 615/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:06:10.244258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:10.356496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ef846-ec32-4a53-963c-909d8581e1cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:10.359087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:10.393030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:10.393632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:10.393949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:10.394245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:10.394973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:10.395310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:10.396908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:10.410049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 614/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:10.415946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:10.426384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:10.531130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf20f-88ac-469b-97f7-16dd5c22aa13 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:10.565129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:10.565756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:10.566034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:10.566308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:10.573720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:10.575491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:06:10.587644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:10.589375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:10.590311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 616/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:06:10.595997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:10.702216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d30c6-334c-4de8-b461-2be421631feb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:10.737517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:10.738124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:10.738401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:10.738671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:10.739438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:10.739778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:10.741228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895e32-f70d-492c-9245-ff7890bfbecb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:10.754244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 615/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:10.761475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:10.830787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:10.868580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f250604-3725-4e0a-9961-87ca31644df6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:10.906402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14646cc2-2cc4-41ce-a130-9b65b6235960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:10.907060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646cc2-2cc4-41ce-a130-9b65b6235960 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:10.907338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646cc2-2cc4-41ce-a130-9b65b6235960 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:10.907613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646cc2-2cc4-41ce-a130-9b65b6235960 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/f669319338ab4c6ea22dead596391789/roles/7cffe6ec486f498693c5d20e2a8375f8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:10.922573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14646cc2-2cc4-41ce-a130-9b65b6235960 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=f669319338ab4c6ea22dead596391789, role_id=7cffe6ec486f498693c5d20e2a8375f8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:10.924000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14646cc2-2cc4-41ce-a130-9b65b6235960 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:10.937302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 617/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:06:10 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/f669319338ab4c6ea22dead596391789/roles/7cffe6ec486f498693c5d20e2a8375f8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:06:12.841706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e6cb-3044-46ad-af81-ee38b73da611 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:12.842267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e6cb-3044-46ad-af81-ee38b73da611 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:12.842267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e6cb-3044-46ad-af81-ee38b73da611 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:12.843597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 616/1233] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:12.849090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:12.849563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:12.849651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:12.868092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:12.869393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:12.942390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 618/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:13.262755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751e6cb-3044-46ad-af81-ee38b73da611 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:13.295163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:13.392453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751e6cb-3044-46ad-af81-ee38b73da611 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:13.428698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:13.429325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:13.429604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:13.429876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:13.440385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:06:13.445022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:13.446116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:13.446987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 617/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Aug 18 11:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:06:13.455330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:13.504876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:13.569089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4204d92-b7ab-462c-8467-451230a5b7fa None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:13.608956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:13.609715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:13.610040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:13.610409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:13.611149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:13.611553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:13.613303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:13.622826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 619/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Aug 18 11:06:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:13.629050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:13.727929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:13.747799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053b3e4-f33a-4e4b-9ecd-439194bd7f70 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:13.785085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:13.785821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:13.786147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:13.786493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:13.800251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:13.801711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:13.819570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:13.820692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:13.826786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:13.826786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:13.826786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:13.826786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 618/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:13.827195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:13.946742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1ab3c-4955-4a5d-a6eb-26627d28608b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:13.959840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:13.989372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:13.989978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:13.990259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:13.990531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:13.991176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:13.991479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:13.992871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:14.007620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 620/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:14.013859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:14.129188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153897d-919e-4a95-b9a1-af0252044af7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:14.167389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:14.168249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:14.168715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:14.168997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:14.172466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:14.177962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:14.179762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:06:14.191664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:14.192799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:14.193763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 619/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:14.200405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:14.307131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552bde0-c294-450b-9217-40710f0d4197 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:14.353040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:14.353801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:14.354085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:14.354486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:14.355341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:14.355789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:14.357476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:14.381284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 621/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Aug 18 11:06:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:14.388497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:14.508071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1814e-9042-4da2-87cf-7e3c1d3ff0bd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:14.551205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:14.551946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:14.552316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:14.552717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:14.629174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:14.637333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:14.638786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:14.759653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 620/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:06:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:06:14.761095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:14.790767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:14.873855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d5d4a-edd2-45c5-8d72-03698df24018 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:14.914031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:14.914031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:14.914031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:14.914031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:14.925067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:14.926679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:14.945620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:14.946827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:14.948441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:14.948441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:14.948441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:14.948441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 622/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:14.955019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:15.065434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:15.071903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5d6ec-e8da-476e-927a-d067856c6ce3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:15.113191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:15.114002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:15.114363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:15.114634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:15.115270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:15.115695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:15.117474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:15.131581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 621/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:15.137390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:15.228068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:15.257587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc023-6c7a-4131-b792-8864189db58e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:15.299438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:15.300273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:15.300661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:15.300930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:15.308528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:15.310021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:06:15.321935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:15.323968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:15.325391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 623/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:06:15.329725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:15.466380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d9f27-8989-4e93-b337-a09c05084e0c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:15.505526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:15.506119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:15.506522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:15.506798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:15.507638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:15.507944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:15.509770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f84c-6b0e-4088-964c-9874d1398878 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:15.523701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 622/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:15.530134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:15.651061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3828-ee9d-468f-8645-070efa1af9c5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:15.691057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:15.691702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:15.692006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:15.692351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:15.693209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] RBAC: Adding query filter params (role.id=7cffe6ec486f498693c5d20e2a8375f8, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=f669319338ab4c6ea22dead596391789) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:15.693492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:15.694993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1980-a418-4b9d-b307-95a3531f7fcd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:15.701592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 624/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:06:15 2022] GET /identity/v3/role_assignments?user.id=f669319338ab4c6ea22dead596391789&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=7cffe6ec486f498693c5d20e2a8375f8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:18.022007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee48537-10ee-4234-b28a-bf845e3bbe41 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:18.022620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee48537-10ee-4234-b28a-bf845e3bbe41 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:18.022620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee48537-10ee-4234-b28a-bf845e3bbe41 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:18.024280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 623/1247] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:18.030606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:18.030999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:18.031379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:18.038738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:18.052742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:18.054353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:18.133529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 625/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:18.441222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee48537-10ee-4234-b28a-bf845e3bbe41 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:18.463109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:18.572453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee48537-10ee-4234-b28a-bf845e3bbe41 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:18.612656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:18.613523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:18.613523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:18.613956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:18.627280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:06:18.633351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:18.634368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:18.635752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 624/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:18.640770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:18.762728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9251d9-73fe-431e-b962-ad10bc789ee8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:18.795422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:18.801517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81f24-0526-452c-8208-66646d607728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:18.802130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81f24-0526-452c-8208-66646d607728 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:18.802450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81f24-0526-452c-8208-66646d607728 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:18.802767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81f24-0526-452c-8208-66646d607728 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:18.803436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81f24-0526-452c-8208-66646d607728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:18.803736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81f24-0526-452c-8208-66646d607728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:18.805131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81f24-0526-452c-8208-66646d607728 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:18.814306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 626/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:18.820248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:18.912727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:18.946465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5b256-2863-4d54-80b4-a2ac345b3ea0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:18.987165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:18.987808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:18.988087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:18.988410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:18.999365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:19.002504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:19.021046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:19.029487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:19.030869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:19.030869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:19.030869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:19.030869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 625/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:19.031253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81f24-0526-452c-8208-66646d607728 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:19.135334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81f24-0526-452c-8208-66646d607728 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:19.174253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:19.174857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:19.175132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:19.175399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:19.176036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:19.176512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:19.178374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:19.193793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 627/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:19.200675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:19.324403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1696ae-722b-471a-85ce-aa36f6916383 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:19.363576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:19.375799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:19.376762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:19.377050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:19.377397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:19.390354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:19.392360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:06:19.405998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:19.407128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:19.408090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 626/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:19.413427 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:19.491937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:19.551709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee9fa9-4878-422f-8e77-633cf924e838 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:19.597617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:19.598198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:19.598535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:19.598816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:19.599552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:19.599884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:19.601425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:19.623677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 628/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Aug 18 11:06:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:19.629986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:19.753078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94183c98-1493-4d6c-8714-17ea17f2b98b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:19.797097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:19.797907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:19.798399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:19.798883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:19.870552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:19.887013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:19.888507 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:20.020390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 627/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:06:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:06:20.030988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:20.162300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797f785-fb85-4309-b727-36bbe5d1222f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:20.176858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:20.205436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:20.206150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:20.206411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:20.206757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:20.218066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:20.219562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:20.235745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:20.236875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:20.237782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:20.237782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:20.237782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:20.237782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 629/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:20.244773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:20.352770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:20.369668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10ace2-fa56-4388-9370-41ec0a5d41a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:20.414999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:20.415428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:20.415704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:20.415972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:20.416792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:20.417604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:20.419070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:20.431856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 628/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:20.438685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:20.547154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae19b36-b092-4b76-a51e-502e5124cac1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:20.586393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:20.587082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:20.587546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:20.587831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:20.596970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:20.598918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:06:20.614257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:20.615652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:20.616771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 630/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 11:06:20.622855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:20.748716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5563c-1f78-4db8-af6f-808f79f5fda5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:20.797730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:20.798467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:20.798749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:20.799019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:20.799726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:20.800029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:20.801529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1158f8-5eb0-445d-9831-4a16b53cfa44 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:20.815602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 629/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:20.822027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:20.849627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:20.929500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fece7-43ba-44ba-b45c-c174c610a227 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:20.966124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:20.966741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:20.967017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:20.967285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:20.968020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=f669319338ab4c6ea22dead596391789) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:20.968365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:20.969821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9acc0-c36c-455e-bb3f-a7b922abf534 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:20.988238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 631/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:06:20 2022] GET /identity/v3/role_assignments?user.id=f669319338ab4c6ea22dead596391789&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:22.872129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5277a-1ded-4234-8a14-8f73b62a20de None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:22.872694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5277a-1ded-4234-8a14-8f73b62a20de None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:22.872694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5277a-1ded-4234-8a14-8f73b62a20de None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:22.874042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 630/1261] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:22.879307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:22.879686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:22.879858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:22.898044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:22.899227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:22.963512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 632/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:23.227039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5277a-1ded-4234-8a14-8f73b62a20de None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:23.238502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:23.332595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5277a-1ded-4234-8a14-8f73b62a20de None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:23.371089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:23.371745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:23.372086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:23.372449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:23.385128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:06:23.390916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:23.391955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:23.393008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 631/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:23.398208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:23.417908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:23.495396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c40358-2f43-4b9f-8e2e-0949d39cb727 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:23.528459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:23.529049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:23.529325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:23.529596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:23.530245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:23.530548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:23.531897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:23.539638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 633/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:23.546133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:23.611459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:23.644846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4622-b25e-4057-9fbe-a412c5d31e79 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:23.678100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:23.678707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:23.678985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:23.679256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:23.689740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:23.691175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:23.705547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:23.706632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:23.707736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:23.707736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:23.707736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:23.707736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 632/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:23.712471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:23.791608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:23.811494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf15a5-b290-44a2-8b52-64d14926fa7f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:23.843825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:23.844459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:23.844737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:23.845005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:23.845639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:23.845944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:23.847293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:23.860015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 634/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:23.865598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:23.967342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58a972-2591-4cde-9112-6b5c2672bef5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:23.988747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:24.003783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.004427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.004703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.004968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.012594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.014022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:06:24.024672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:24.025778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:24.026827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 633/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:24.031750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:24.129539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc62076-6dbc-4e24-8783-c7a8ebf032e7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:24.162311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.162898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.163173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.163441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.164154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:24.164483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.165895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.174557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:24.183768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 635/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Aug 18 11:06:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:24.189539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:24.287827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c390f-e00e-4d58-901f-92930983a543 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:24.321759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.322387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.322669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.323696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.338704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.341160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:24.359972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:24.361072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:24.362364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:24.362364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:24.362364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:24.362364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 634/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:24.367512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:24.466382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d2a8b-8b5b-46e9-95c9-6c7de3177444 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:24.499669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.500539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.500814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.501083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.501709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:24.502008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.503354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.514387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 636/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:24.519971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:24.551527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:24.621680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4941845-03d5-44e7-8159-80bda5c474b9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:24.654375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.654986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.655266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.655541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.661993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.663425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:06:24.675390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:24.676537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:24.677715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 635/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:24.683039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:24.738723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:24.779629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133866d2-5b61-4a72-8d02-bbbfd8d2b282 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:24.813020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.813604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.813877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.814141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.814814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:24.815110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.816542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47351860-0b3e-4a7c-b265-ddc1ce92d303 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.828046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 637/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:24.834031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:24.923799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:24.932849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1fc8c-9e9b-43b0-a9a2-64dfa3a20e54 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:24.965692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51c72f-6ab7-4f7f-a4d5-2d2d06a6fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:24.966329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51c72f-6ab7-4f7f-a4d5-2d2d06a6fcc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:24.966607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51c72f-6ab7-4f7f-a4d5-2d2d06a6fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:24.966882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51c72f-6ab7-4f7f-a4d5-2d2d06a6fcc9 None admin] PATH_INFO: `/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/f669319338ab4c6ea22dead596391789/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:24.968808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51c72f-6ab7-4f7f-a4d5-2d2d06a6fcc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdd785798ec4a6ba0c5ebd91d110098, user_id=f669319338ab4c6ea22dead596391789, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:24.970236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51c72f-6ab7-4f7f-a4d5-2d2d06a6fcc9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:24.982440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 636/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Aug 18 11:06:24 2022] PUT /identity/v3/projects/0bdd785798ec4a6ba0c5ebd91d110098/users/f669319338ab4c6ea22dead596391789/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:06:26.904855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53e58-e1a3-430e-bcaa-35f06084f078 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:26.905421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53e58-e1a3-430e-bcaa-35f06084f078 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:26.905421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53e58-e1a3-430e-bcaa-35f06084f078 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:26.906770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 638/1274] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:26.911767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:26.912212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:26.912330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:26.930632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:26.931827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:26.998213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 637/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:27.268360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f53e58-e1a3-430e-bcaa-35f06084f078 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:27.379144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f53e58-e1a3-430e-bcaa-35f06084f078 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:27.400737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:27.418765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:27.419385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:27.419666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:27.419939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:27.429891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 11:06:27.434612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:27.436271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:27.437398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 639/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Aug 18 11:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:27.441763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:27.521356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:27.540437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0348e-a0e9-4fa6-be29-204c8ce15046 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:27.573717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:27.574311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:27.574590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:27.574861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:27.575520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:27.575824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:27.577224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:27.585039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 638/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Aug 18 11:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:27.591323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:27.688022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367933af-4fd6-41a7-ae2d-b6b70ab24ed7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:27.721347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:27.721948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:27.722220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:27.722486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:27.732133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:27.733580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:27.747762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:27.752285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:27.753440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:27.753440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:27.753440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:27.753440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 640/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:27.754935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:27.852808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c33c40-18ed-483c-89d8-b273a571a77e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:27.885274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:27.886492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:27.886766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:27.887034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:27.887664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:27.887959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:27.889335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:27.900543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 639/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:27.906103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:27.936302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:28.008518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670828a-7ba5-47ef-b346-eebf87c6e886 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:28.042002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:28.042614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:28.043051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:28.043320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:28.050593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:28.052011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 11:06:28.063277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:28.064429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:28.065595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 641/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:28.070141 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:28.074664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:28.169245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f6d4-b319-48de-9773-d09108523699 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:28.202969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:28.203565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:28.203845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:28.204120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:28.204882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:28.205188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:28.206616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:28.224053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 640/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Aug 18 11:06:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:28.230429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:28.320228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:28.332472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105f250-3753-4a0b-9edd-6f17cda087d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:28.368028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:28.368656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:28.368935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:28.369206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:28.434875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:28.436270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:28.520428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:28.531065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 642/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Aug 18 11:06:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 11:06:28.537128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:28.597438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:28.635788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdcef7-cf6d-4598-9668-617760122dc3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:28.669325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:28.669925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:28.670199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:28.670471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:28.680054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:28.681511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 11:06:28.695394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:28.696555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 11:06:28.697676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 11:06:28.697676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 11:06:28.697676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:28.697676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 641/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:28.704031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:28.801762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153cc50-87b4-4fb6-a809-6241c0fcdb43 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:28.834609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:28.835194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:28.835469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:28.835739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:28.836399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:28.836700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:28.838046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:28.849451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 643/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Aug 18 11:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:28.855104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:28.953985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5976c-3150-4459-9f1e-0ff81e555030 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:28.968362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:28.988665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:28.989279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:28.989558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:28.989833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:28.996470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:28.997935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 11:06:29.009244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:29.010153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 18 11:06:29.011652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 642/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Aug 18 11:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:29.017034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:29.070957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:29.122949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab55b9-dbf4-44cf-85e3-96924b651f2a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:29.157244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:29.157842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:29.158119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:29.158389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:29.159066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:29.159367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:29.160824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aef91f-6259-4c03-b974-756a3777f565 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:29.172462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 644/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Aug 18 11:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:29.178603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:29.285412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef271d-3d72-4065-a5fe-c1c61b509dac None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:29.318150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:29.318754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:29.319075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:29.319381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:29.320214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] RBAC: Adding query filter params (role.id=cd03b4f182254618b6e0a071d21dd304, scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098, user.id=f669319338ab4c6ea22dead596391789) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:29.320521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:29.321955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a15c-c6b1-4bf3-8b91-82e1aebc8137 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:29.329217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 643/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Aug 18 11:06:29 2022] GET /identity/v3/role_assignments?user.id=f669319338ab4c6ea22dead596391789&scope.project.id=0bdd785798ec4a6ba0c5ebd91d110098&role.id=cd03b4f182254618b6e0a071d21dd304 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:31.246192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea2e3f-0a56-42d5-b821-b09fe004b273 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:31.246746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea2e3f-0a56-42d5-b821-b09fe004b273 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:31.246746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea2e3f-0a56-42d5-b821-b09fe004b273 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:31.248069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 645/1288] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:31.253496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:31.253929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:31.254032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:31.272241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:31.273426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:31.302531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:31.342763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 644/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:06:31.609090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea2e3f-0a56-42d5-b821-b09fe004b273 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:31.709028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea2e3f-0a56-42d5-b821-b09fe004b273 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:31.719392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:31.742173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:31.742789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:31.743067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:31.743338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:31.743913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:31.745847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:31.754569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:31.755870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:31.756971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 646/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:06:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:31.762116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:31.858160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cad9e-4541-4763-a718-76190043d8ae None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:31.893176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:31.893709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:31.893989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:31.894262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:31.894892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:31.895193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:31.896591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63042fec-e52e-4331-8c5e-7892d9b15fdd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:31.901349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 645/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Aug 18 11:06:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:31.906983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:32.004870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ac7df-ff65-4c39-b29e-8a5045e90b6c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:32.037243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:32.037857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:32.038130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:32.038397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:32.039007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:32.039303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:32.040687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64618e06-8bfa-4114-a38a-ff8a8d03060a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:32.045560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 647/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Aug 18 11:06:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:33.918217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d40b3-e121-4348-8124-5a6a37baf080 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:33.918768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d40b3-e121-4348-8124-5a6a37baf080 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:33.918768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d40b3-e121-4348-8124-5a6a37baf080 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:33.920103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 646/1293] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:33.925277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac554-ce1a-4f55-a0b3-3d249687a076 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:33.925733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac554-ce1a-4f55-a0b3-3d249687a076 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:33.925829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac554-ce1a-4f55-a0b3-3d249687a076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:33.944066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-624ac554-ce1a-4f55-a0b3-3d249687a076 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:33.945282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-624ac554-ce1a-4f55-a0b3-3d249687a076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:33.974309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:34.013218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 648/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:34.295060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d40b3-e121-4348-8124-5a6a37baf080 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:34.329423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:34.418857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d40b3-e121-4348-8124-5a6a37baf080 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:34.452178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891280a-88e2-47d1-aa83-d7174516c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:34.452800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891280a-88e2-47d1-aa83-d7174516c375 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:34.453082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891280a-88e2-47d1-aa83-d7174516c375 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:34.453357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891280a-88e2-47d1-aa83-d7174516c375 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:34.454002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891280a-88e2-47d1-aa83-d7174516c375 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:34.455369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891280a-88e2-47d1-aa83-d7174516c375 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:34.469099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 647/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Aug 18 11:06:34 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:06:36.383551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726a49f-96e5-4e07-a790-26b11d26af21 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:36.384060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726a49f-96e5-4e07-a790-26b11d26af21 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:36.384060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726a49f-96e5-4e07-a790-26b11d26af21 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:36.386039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 649/1296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:36.391271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857420af-3086-4633-936c-325482b984d0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:36.391694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857420af-3086-4633-936c-325482b984d0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:36.391811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857420af-3086-4633-936c-325482b984d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:36.410099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-857420af-3086-4633-936c-325482b984d0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:36.411275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-857420af-3086-4633-936c-325482b984d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:36.482004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 648/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 11:06:36.748117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726a49f-96e5-4e07-a790-26b11d26af21 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:36.846886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726a49f-96e5-4e07-a790-26b11d26af21 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:36.879670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:36.880304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:36.880583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:36.880854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:36.881426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:36.882772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:36.891782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:36.892948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:36.893810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 650/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:06:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:36.898784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857420af-3086-4633-936c-325482b984d0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:36.916666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:36.997424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857420af-3086-4633-936c-325482b984d0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:37.031443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:37.032048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:37.032366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:37.032641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:37.033270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:37.033570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:37.034929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:37.039927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 649/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Aug 18 11:06:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:06:37.045884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:37.078250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:37.144890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304662d-7576-4331-a09d-2081722d4b01 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:37.177285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:37.177880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:37.178160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:37.178433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:37.179119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] RBAC: Adding query filter params (interface=public, service_id=101c7c307fd04c07a986b5c986370592, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:37.179426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:37.180817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612aad66-68d9-42e8-8522-5c5158025276 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:37.186218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 651/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Aug 18 11:06:37 2022] GET /identity/v3/endpoints?service_id=101c7c307fd04c07a986b5c986370592&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:37.191909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:37.272090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:37.290206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbab45b-e781-47ac-a649-24284fead819 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:37.325308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e41cb-a671-4b58-8614-617ae19351e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:37.325907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e41cb-a671-4b58-8614-617ae19351e5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:37.326191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e41cb-a671-4b58-8614-617ae19351e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:37.326469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e41cb-a671-4b58-8614-617ae19351e5 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:37.327072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e41cb-a671-4b58-8614-617ae19351e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:37.328465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e41cb-a671-4b58-8614-617ae19351e5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:37.335609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 650/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Aug 18 11:06:37 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:06:39.220642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7870f8-229c-4318-89b3-33b565eb07de None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:39.221197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7870f8-229c-4318-89b3-33b565eb07de None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:39.221197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7870f8-229c-4318-89b3-33b565eb07de None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:39.222537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 652/1302] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:06:39.227548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:39.227963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:39.228075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:39.246164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:06:39.247326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:06:39.312216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 651/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:06:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 11:06:39.589000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7870f8-229c-4318-89b3-33b565eb07de None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:39.681168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:39.688700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7870f8-229c-4318-89b3-33b565eb07de None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:39.720605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:39.721217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:39.721492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:39.721757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:39.722327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:39.723671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 11:06:39.732800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 11:06:39.733975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Aug 18 11:06:39.735731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 653/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Aug 18 11:06:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 11:06:39.740088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:39.815133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:39.838100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54ff49-acf4-4123-98a5-6985b5733f3b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:39.873073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:39.873607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:39.873887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:39.874161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:39.874788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:06:39.875090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:39.876745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081fdbf-ba64-4200-b21b-574a8d7b0995 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:39.881712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 652/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Aug 18 11:06:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:06:39.887816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:06:39.985085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea10be-6c69-4dc4-adf3-fbb93b9257e6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:06:40.018451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a67394-6d9f-4956-b7cc-8e6bbf543057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:06:40.019051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a67394-6d9f-4956-b7cc-8e6bbf543057 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:06:40.019330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a67394-6d9f-4956-b7cc-8e6bbf543057 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:06:40.019601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a67394-6d9f-4956-b7cc-8e6bbf543057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:06:40.020293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a67394-6d9f-4956-b7cc-8e6bbf543057 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:06:40.022527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a67394-6d9f-4956-b7cc-8e6bbf543057 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:06:40.044542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 654/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:06:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:06:49.878753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=30 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:06:50.039370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:44.312796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc419ef-4d0b-42d4-9b68-d0edb27a4958 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:44.313679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc419ef-4d0b-42d4-9b68-d0edb27a4958 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:44.314111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc419ef-4d0b-42d4-9b68-d0edb27a4958 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:44.319454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 653/1307] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:07:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:44.320871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb865da-1f7e-48b5-8185-ea0e19f8e9bd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:44.321803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb865da-1f7e-48b5-8185-ea0e19f8e9bd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:44.322247 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb865da-1f7e-48b5-8185-ea0e19f8e9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:44.467560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7bb865da-1f7e-48b5-8185-ea0e19f8e9bd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:44.469063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7bb865da-1f7e-48b5-8185-ea0e19f8e9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:44.670649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 655/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:07:44.976750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d982c9b-d67d-45d0-902a-a06f50ea6345 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:44.977031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d982c9b-d67d-45d0-902a-a06f50ea6345 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:44.977308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d982c9b-d67d-45d0-902a-a06f50ea6345 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:44.978803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 654/1309] 10.222.0.16 () {54 vars in 965 bytes} [Thu Aug 18 11:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:44.985863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda86f8-5e33-46d5-87f0-a41a40ab8a5c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:44.986151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda86f8-5e33-46d5-87f0-a41a40ab8a5c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:44.986429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda86f8-5e33-46d5-87f0-a41a40ab8a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:45.085599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-adda86f8-5e33-46d5-87f0-a41a40ab8a5c None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:45.086855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-adda86f8-5e33-46d5-87f0-a41a40ab8a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:45.109067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:45.169210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 656/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Thu Aug 18 11:07:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:07:45.178206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d982c9b-d67d-45d0-902a-a06f50ea6345 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:07:45.180021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6d982c9b-d67d-45d0-902a-a06f50ea6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:45.296245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d982c9b-d67d-45d0-902a-a06f50ea6345 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:07:45.377286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:07:45.378983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:45.379331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:45.379669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:45.381390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:45.441402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:45.506756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:07:45.508222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23338ef-2a46-4396-903c-9387b47e71b6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:07:45.529438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 655/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:07:47.800732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca80d-3d53-41e4-9662-10f667c27f1d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:47.801263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca80d-3d53-41e4-9662-10f667c27f1d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:47.801263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca80d-3d53-41e4-9662-10f667c27f1d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:47.802667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 657/1312] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:47.808093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6f45e-a567-4dc3-bc1b-3e49810c6bf5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:47.808534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6f45e-a567-4dc3-bc1b-3e49810c6bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:47.808671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6f45e-a567-4dc3-bc1b-3e49810c6bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:47.880522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-74a6f45e-a567-4dc3-bc1b-3e49810c6bf5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:47.881738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74a6f45e-a567-4dc3-bc1b-3e49810c6bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:47.993670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 656/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:07:48.309524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28bb82-0fb6-4a03-a925-b3de1b6fa401 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:48.309986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28bb82-0fb6-4a03-a925-b3de1b6fa401 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:48.310269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28bb82-0fb6-4a03-a925-b3de1b6fa401 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:48.311636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 658/1314] 10.222.0.16 () {54 vars in 965 bytes} [Thu Aug 18 11:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:48.318262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56ae0-5628-443f-8b25-faa186211e0c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:48.318621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56ae0-5628-443f-8b25-faa186211e0c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:48.318964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56ae0-5628-443f-8b25-faa186211e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:48.331340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:48.398984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6da56ae0-5628-443f-8b25-faa186211e0c None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:48.400400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6da56ae0-5628-443f-8b25-faa186211e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:48.480640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 657/1315] 10.222.0.16 () {58 vars in 1062 bytes} [Thu Aug 18 11:07:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:07:48.490235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28bb82-0fb6-4a03-a925-b3de1b6fa401 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:07:48.491837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4a28bb82-0fb6-4a03-a925-b3de1b6fa401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:48.608071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28bb82-0fb6-4a03-a925-b3de1b6fa401 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:07:48.687816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:48.690820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:07:48.691639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:48.692117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:48.692624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:48.694377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:48.814643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:07:48.816082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e17-2924-46e2-bebf-932a222516b4 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:07:48.834173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 659/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:07:51.468896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50c6e-01ea-4736-a3af-83248bdd3787 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:51.469449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50c6e-01ea-4736-a3af-83248bdd3787 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:51.469449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50c6e-01ea-4736-a3af-83248bdd3787 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:51.470841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 658/1317] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:51.476331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99863f-aad5-46c1-829e-47455b5e9464 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:51.476942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99863f-aad5-46c1-829e-47455b5e9464 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:51.476942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99863f-aad5-46c1-829e-47455b5e9464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:51.547957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5c99863f-aad5-46c1-829e-47455b5e9464 None None] MFA Rules not processed for user `b42994e086454bed9832f0d860d67c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:51.549194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5c99863f-aad5-46c1-829e-47455b5e9464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:51.603202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:51.654175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 660/1318] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:07:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:07:51.975173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50c6e-01ea-4736-a3af-83248bdd3787 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:07:51.976789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3be50c6e-01ea-4736-a3af-83248bdd3787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:52.095819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50c6e-01ea-4736-a3af-83248bdd3787 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:07:52.099422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:52.191775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:07:52.192884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:52.193405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:52.193917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:52.197240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:52.325487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:07:52.326835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c937acc-66cc-4f4d-8d28-7b5776149bbb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:07:52.345196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 659/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:07:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:07:54.854859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a628e-7c99-42cc-9c63-1a50d3288ccf None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:54.855411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a628e-7c99-42cc-9c63-1a50d3288ccf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:54.855411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a628e-7c99-42cc-9c63-1a50d3288ccf None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:54.856842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 661/1320] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:54.861874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82253141-42c6-4499-98f2-4280e24a8402 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:54.862340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82253141-42c6-4499-98f2-4280e24a8402 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:54.862455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82253141-42c6-4499-98f2-4280e24a8402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:54.904369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:54.930830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-82253141-42c6-4499-98f2-4280e24a8402 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:54.932052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-82253141-42c6-4499-98f2-4280e24a8402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:55.037636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 660/1321] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:07:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:07:55.339113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a628e-7c99-42cc-9c63-1a50d3288ccf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:07:55.341257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-869a628e-7c99-42cc-9c63-1a50d3288ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:55.444621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a628e-7c99-42cc-9c63-1a50d3288ccf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:07:55.518421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:07:55.519015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:55.519289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:55.519556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:55.521183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:55.529211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:55.629216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:07:55.630541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caa2e2-d3fa-444f-a66b-aa3ca63d9291 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:07:55.650076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 662/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:07:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:07:58.339276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2414e-a582-4b04-a0e0-37fcac1b0280 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:58.339841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2414e-a582-4b04-a0e0-37fcac1b0280 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:58.339841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2414e-a582-4b04-a0e0-37fcac1b0280 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:58.341157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 661/1323] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:07:58.346552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d224-74dd-443c-828f-aa49e932102f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:58.347025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d224-74dd-443c-828f-aa49e932102f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:58.347133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d224-74dd-443c-828f-aa49e932102f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:58.415576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-64e0d224-74dd-443c-828f-aa49e932102f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:07:58.416811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-64e0d224-74dd-443c-828f-aa49e932102f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:58.476431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:58.523853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 663/1324] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:07:58.809643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2414e-a582-4b04-a0e0-37fcac1b0280 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:07:58.811285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0db2414e-a582-4b04-a0e0-37fcac1b0280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:58.917011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2414e-a582-4b04-a0e0-37fcac1b0280 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:07:58.929996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:07:59.001172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:07:59.001782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:07:59.002060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:07:59.002601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:07:59.004264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:07:59.117272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:07:59.118799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46543d34-e0d1-4755-9068-2b17978636b3 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:07:59.138840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 662/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:01.150371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1482b8-87e8-43a0-aa74-3f988661720a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:01.150911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1482b8-87e8-43a0-aa74-3f988661720a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:01.150911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1482b8-87e8-43a0-aa74-3f988661720a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:01.152513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 664/1326] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:01.157717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a397ba-0558-493f-9531-1af3c3084272 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:01.158145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a397ba-0558-493f-9531-1af3c3084272 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:01.158249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a397ba-0558-493f-9531-1af3c3084272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:01.212593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:01.231868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-70a397ba-0558-493f-9531-1af3c3084272 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:01.233124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70a397ba-0558-493f-9531-1af3c3084272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:01.332756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 663/1327] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:01.617711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1482b8-87e8-43a0-aa74-3f988661720a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:01.619309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fd1482b8-87e8-43a0-aa74-3f988661720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:01.740036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1482b8-87e8-43a0-aa74-3f988661720a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:01.818172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:01.818780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:01.819053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:01.819322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:01.820964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:01.838519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:01.934674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:01.936097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebd1d0-a3f4-4270-8ee7-99f86276e451 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:01.962302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 665/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:03.942396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c7031-905c-4459-8313-bd8bb25d07dd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:03.942990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c7031-905c-4459-8313-bd8bb25d07dd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:03.942990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c7031-905c-4459-8313-bd8bb25d07dd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:03.944317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 664/1329] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:03.950635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e03384-0d26-41ad-be13-e70148ef790f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:03.951159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e03384-0d26-41ad-be13-e70148ef790f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:03.951159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e03384-0d26-41ad-be13-e70148ef790f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:04.022348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c7e03384-0d26-41ad-be13-e70148ef790f None None] MFA Rules not processed for user `b42994e086454bed9832f0d860d67c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:04.023584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c7e03384-0d26-41ad-be13-e70148ef790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:04.092766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:04.126017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 666/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:04.414333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c7031-905c-4459-8313-bd8bb25d07dd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:04.415885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bb0c7031-905c-4459-8313-bd8bb25d07dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:04.524091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c7031-905c-4459-8313-bd8bb25d07dd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:04.544895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:04.601240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:04.601773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:04.602048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:04.602317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:04.603930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:04.716461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:04.717788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ced84-8cd3-4aab-aa98-5d581f5f10ca service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:04.736853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 665/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:07.103478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407d033-8aef-440a-9a6d-393725d03059 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:07.104012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407d033-8aef-440a-9a6d-393725d03059 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:07.104012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407d033-8aef-440a-9a6d-393725d03059 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:07.105459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 667/1332] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:07.110893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe638c-b5f1-43ee-8e58-5755c9493021 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:07.111367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe638c-b5f1-43ee-8e58-5755c9493021 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:07.111463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe638c-b5f1-43ee-8e58-5755c9493021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:07.180239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:07.183846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1fe638c-b5f1-43ee-8e58-5755c9493021 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:07.186028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1fe638c-b5f1-43ee-8e58-5755c9493021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:07.292604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 666/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:07.580612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407d033-8aef-440a-9a6d-393725d03059 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:07.582155 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6407d033-8aef-440a-9a6d-393725d03059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:07.689576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407d033-8aef-440a-9a6d-393725d03059 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:07.767093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:07.767695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:07.767978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:07.768270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:07.770041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:07.800370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:07.889785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:07.891238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbb7c1-c520-4e2f-bd53-cc97d8035da3 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:07.909929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 668/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:09.871970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53269249-6f7c-4fc7-97c4-c85ae0718910 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:09.872559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53269249-6f7c-4fc7-97c4-c85ae0718910 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:09.872559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53269249-6f7c-4fc7-97c4-c85ae0718910 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:09.873934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 667/1335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:09.879399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb508b-eb1d-446a-8045-a28a73354452 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:09.879847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb508b-eb1d-446a-8045-a28a73354452 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:09.879972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb508b-eb1d-446a-8045-a28a73354452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:09.944827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f8eb508b-eb1d-446a-8045-a28a73354452 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:09.945968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f8eb508b-eb1d-446a-8045-a28a73354452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:10.020215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:10.043969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 669/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:10.330773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53269249-6f7c-4fc7-97c4-c85ae0718910 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:10.332351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53269249-6f7c-4fc7-97c4-c85ae0718910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:10.438266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53269249-6f7c-4fc7-97c4-c85ae0718910 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:10.470561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:10.516959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:10.517556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:10.517830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:10.518094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:10.519724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:10.628998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:10.630335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff2680-255f-44ba-a8a6-2b1b62ff6a13 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:10.648065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 668/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:08:13.078563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f4f8-b93a-493e-878e-67e422d108a2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:13.079133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f4f8-b93a-493e-878e-67e422d108a2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:13.079133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f4f8-b93a-493e-878e-67e422d108a2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:13.080481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 670/1338] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:13.085618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b239f-d47e-4b79-ad15-c762372d8f34 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:13.086077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b239f-d47e-4b79-ad15-c762372d8f34 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:13.086172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b239f-d47e-4b79-ad15-c762372d8f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:13.155025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0e7b239f-d47e-4b79-ad15-c762372d8f34 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:13.156327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0e7b239f-d47e-4b79-ad15-c762372d8f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:13.164472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:13.263578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 669/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:13.563540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1f4f8-b93a-493e-878e-67e422d108a2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:13.565111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f4d1f4f8-b93a-493e-878e-67e422d108a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:13.670253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1f4f8-b93a-493e-878e-67e422d108a2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:13.748795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:13.749392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:13.749666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:13.749933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:13.751667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:13.793441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:13.869287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:13.871554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de0f79-138a-45cc-8709-3a081ce4362a service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:13.893842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 671/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:17.943177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839381-fc78-4de7-9b5b-099f18e0ce9a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:17.943608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839381-fc78-4de7-9b5b-099f18e0ce9a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:17.943776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839381-fc78-4de7-9b5b-099f18e0ce9a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:17.945146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 670/1341] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:17.951167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e48ae-c0e4-4a41-98a7-0f458bee8ea7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:17.951767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e48ae-c0e4-4a41-98a7-0f458bee8ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:17.951767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e48ae-c0e4-4a41-98a7-0f458bee8ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:18.021806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-356e48ae-c0e4-4a41-98a7-0f458bee8ea7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:18.023004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-356e48ae-c0e4-4a41-98a7-0f458bee8ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:18.111890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:18.123899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 672/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:18.409463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57839381-fc78-4de7-9b5b-099f18e0ce9a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:18.411047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57839381-fc78-4de7-9b5b-099f18e0ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:18.522625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57839381-fc78-4de7-9b5b-099f18e0ce9a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:18.562685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:18.601292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:18.602082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:18.602516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:18.602940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:18.606231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:18.721803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:18.723242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f57f3-cd89-4c2c-b55a-02999d451b17 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:18.741829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 671/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:21.386819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ead341-0f3f-4510-8e50-723caa9ebeeb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:21.387370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ead341-0f3f-4510-8e50-723caa9ebeeb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:21.387370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ead341-0f3f-4510-8e50-723caa9ebeeb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:21.389044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 673/1344] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:21.394713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f338b4b-4119-4fc9-a389-86dd50343551 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:21.395113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f338b4b-4119-4fc9-a389-86dd50343551 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:21.395308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f338b4b-4119-4fc9-a389-86dd50343551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:21.469880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2f338b4b-4119-4fc9-a389-86dd50343551 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:21.471093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2f338b4b-4119-4fc9-a389-86dd50343551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:21.489225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:21.572149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 672/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:21.861363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ead341-0f3f-4510-8e50-723caa9ebeeb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:21.862942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-41ead341-0f3f-4510-8e50-723caa9ebeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:21.969876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ead341-0f3f-4510-8e50-723caa9ebeeb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:22.052910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:22.053517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:22.053792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:22.054060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:22.055659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:22.106531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:22.176317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:22.177745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0e5f6-3118-438a-b7ac-e72ff84aec2d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:22.196954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 674/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:26.501745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1102ca-4d19-407a-8740-e73c0f80e53e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:26.502274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1102ca-4d19-407a-8740-e73c0f80e53e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:26.502380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1102ca-4d19-407a-8740-e73c0f80e53e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:26.503970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 673/1347] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:26.510045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738fdd-067e-46d5-92f8-8716fcba18e4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:26.510503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738fdd-067e-46d5-92f8-8716fcba18e4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:26.511299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738fdd-067e-46d5-92f8-8716fcba18e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:26.602834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-75738fdd-067e-46d5-92f8-8716fcba18e4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:26.604165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-75738fdd-067e-46d5-92f8-8716fcba18e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:26.712003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:26.715240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 675/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:27.002314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1102ca-4d19-407a-8740-e73c0f80e53e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:27.004178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cb1102ca-4d19-407a-8740-e73c0f80e53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:27.125307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1102ca-4d19-407a-8740-e73c0f80e53e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:27.179380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:27.210389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:27.210999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:27.211275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:27.211564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:27.213218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:27.331169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:27.333077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554ed7-42fb-4f77-ba94-220ae9e1b2f0 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:27.353316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 674/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:32.694684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df59af9-7119-4f55-891e-d02ea4c54f5f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:32.695136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df59af9-7119-4f55-891e-d02ea4c54f5f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:32.695538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df59af9-7119-4f55-891e-d02ea4c54f5f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:32.697589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 676/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:32.704314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca626f2a-38bb-4f13-b899-768b8cde3e37 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:32.704877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca626f2a-38bb-4f13-b899-768b8cde3e37 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:32.705030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca626f2a-38bb-4f13-b899-768b8cde3e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:32.812413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ca626f2a-38bb-4f13-b899-768b8cde3e37 None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:32.813629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca626f2a-38bb-4f13-b899-768b8cde3e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:32.837570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:32.986483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 675/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:08:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:37.606342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54841-1efd-4a9f-a8f9-5a34b6938a71 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:37.606868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54841-1efd-4a9f-a8f9-5a34b6938a71 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:37.606868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54841-1efd-4a9f-a8f9-5a34b6938a71 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:37.608152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 677/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:37.613445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad25a0-f5ec-452a-a32d-dfc86f098765 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:37.613910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad25a0-f5ec-452a-a32d-dfc86f098765 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:37.614014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad25a0-f5ec-452a-a32d-dfc86f098765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:37.719533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-50ad25a0-f5ec-452a-a32d-dfc86f098765 None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:37.720816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-50ad25a0-f5ec-452a-a32d-dfc86f098765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:37.822209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:37.869195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 676/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:08:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:39.328478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cbfaf-01c7-48bd-834e-d18a55a80313 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:39.329134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cbfaf-01c7-48bd-834e-d18a55a80313 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:39.329134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cbfaf-01c7-48bd-834e-d18a55a80313 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:39.330259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 678/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:08:39.335621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142b012-c7b6-405b-804c-07e42f6b46de None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:39.336211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142b012-c7b6-405b-804c-07e42f6b46de None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:39.336211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142b012-c7b6-405b-804c-07e42f6b46de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:39.433337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5142b012-c7b6-405b-804c-07e42f6b46de None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:39.435110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5142b012-c7b6-405b-804c-07e42f6b46de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:39.560266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 677/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:43.367045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e389c8a-0a70-4103-97f2-5a4b00a2b613 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:43.367624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e389c8a-0a70-4103-97f2-5a4b00a2b613 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:43.367624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e389c8a-0a70-4103-97f2-5a4b00a2b613 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:43.369073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 679/1356] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:43.375402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcea520-b6df-41f3-aae4-fbc3f2a1dd14 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:43.375904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcea520-b6df-41f3-aae4-fbc3f2a1dd14 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:43.376236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcea520-b6df-41f3-aae4-fbc3f2a1dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:43.415816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:43.462393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9dcea520-b6df-41f3-aae4-fbc3f2a1dd14 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:43.463055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9dcea520-b6df-41f3-aae4-fbc3f2a1dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:43.617025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 678/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:08:43.980959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ef44-c7d0-4b85-9441-a4837647afd6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:43.981817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ef44-c7d0-4b85-9441-a4837647afd6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:43.982478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ef44-c7d0-4b85-9441-a4837647afd6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:43.984451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 680/1358] 10.222.0.16 () {54 vars in 962 bytes} [Thu Aug 18 11:08:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:43.994407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f26822-7247-4c9b-a253-7bb6882a9d8f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:43.995087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f26822-7247-4c9b-a253-7bb6882a9d8f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:43.995870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f26822-7247-4c9b-a253-7bb6882a9d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:44.093413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-13f26822-7247-4c9b-a253-7bb6882a9d8f None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:44.094883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-13f26822-7247-4c9b-a253-7bb6882a9d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:44.156472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:44.253668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 679/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Aug 18 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:08:44.262644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ef44-c7d0-4b85-9441-a4837647afd6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:44.264518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-92f9ef44-c7d0-4b85-9441-a4837647afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:44.443388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ef44-c7d0-4b85-9441-a4837647afd6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:44.535236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:44.586213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:44.586922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:44.587231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:44.587564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:44.591969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:44.704134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:44.706078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503fc8f-451e-4f64-9d21-f63d04d58578 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:44.729313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 681/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:47.038307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe6415-b274-4e89-8926-876b9075aae1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:47.038769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe6415-b274-4e89-8926-876b9075aae1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:47.039122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe6415-b274-4e89-8926-876b9075aae1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:47.041072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 680/1361] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:47.046702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bedd9-ece8-4422-9f77-c942af0081ab None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:47.047283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bedd9-ece8-4422-9f77-c942af0081ab None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:47.047283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bedd9-ece8-4422-9f77-c942af0081ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:47.129780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-134bedd9-ece8-4422-9f77-c942af0081ab None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:47.131429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-134bedd9-ece8-4422-9f77-c942af0081ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:47.186226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:47.248690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 682/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:47.570943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe6415-b274-4e89-8926-876b9075aae1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:47.572697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75fe6415-b274-4e89-8926-876b9075aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:47.779194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe6415-b274-4e89-8926-876b9075aae1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:47.790711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:47.935295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:47.935993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:47.936663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:47.937267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:47.940575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:48.077895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:48.079967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47812e2f-3e56-4dda-9290-dc3633609a2c service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:48.106271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 681/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:08:50.476354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e5e86-5ecb-41e9-afe8-6408b7089b5e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:50.476948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e5e86-5ecb-41e9-afe8-6408b7089b5e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:50.476948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e5e86-5ecb-41e9-afe8-6408b7089b5e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:50.478268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 683/1364] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:50.483740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54458a1-4822-4550-86e4-562f8f83810f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:50.484274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54458a1-4822-4550-86e4-562f8f83810f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:50.484374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54458a1-4822-4550-86e4-562f8f83810f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:50.502612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:50.575145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e54458a1-4822-4550-86e4-562f8f83810f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:50.576493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e54458a1-4822-4550-86e4-562f8f83810f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:50.717158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 682/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:08:51.030226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1a13a-fb4e-4a3d-98da-4dc0882e27e6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:51.030869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1a13a-fb4e-4a3d-98da-4dc0882e27e6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:51.031298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1a13a-fb4e-4a3d-98da-4dc0882e27e6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:51.033151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 684/1366] 10.222.0.16 () {54 vars in 962 bytes} [Thu Aug 18 11:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:51.039152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80abd17-c280-434d-b64a-2e1b7b5c21ed None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:51.039524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80abd17-c280-434d-b64a-2e1b7b5c21ed None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:51.039801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80abd17-c280-434d-b64a-2e1b7b5c21ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:51.132993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c80abd17-c280-434d-b64a-2e1b7b5c21ed None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:51.134678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c80abd17-c280-434d-b64a-2e1b7b5c21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:51.180166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:51.304622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 683/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Aug 18 11:08:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:51.314032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1a13a-fb4e-4a3d-98da-4dc0882e27e6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:51.316724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-81c1a13a-fb4e-4a3d-98da-4dc0882e27e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:51.501373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1a13a-fb4e-4a3d-98da-4dc0882e27e6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:51.525873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:51.638309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:51.638836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:51.639195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:51.639586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:51.641524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:51.773576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:51.775247 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf311da-05d3-42c0-88d4-6064773166ee service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:51.796303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 685/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:52.310471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf76681-9b37-45eb-a5d7-bab2229c66fc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:52.311030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf76681-9b37-45eb-a5d7-bab2229c66fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:52.311030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf76681-9b37-45eb-a5d7-bab2229c66fc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:52.312646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 684/1369] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:52.318104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb99f5a-22ef-4eac-a07b-04e8c5a27fa1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:52.318408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb99f5a-22ef-4eac-a07b-04e8c5a27fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:52.318650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb99f5a-22ef-4eac-a07b-04e8c5a27fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:52.371469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:52.424739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1fb99f5a-22ef-4eac-a07b-04e8c5a27fa1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:52.426370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1fb99f5a-22ef-4eac-a07b-04e8c5a27fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:52.556738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 686/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:53.177899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a7d65-a88f-4950-a011-0a6062b27243 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:53.177899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a7d65-a88f-4950-a011-0a6062b27243 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:53.177899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a7d65-a88f-4950-a011-0a6062b27243 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:53.177899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 685/1371] 10.222.0.16 () {54 vars in 964 bytes} [Thu Aug 18 11:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:53.184058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086d80-9756-4e0e-a126-0444829c3435 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:53.184614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086d80-9756-4e0e-a126-0444829c3435 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:53.185164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086d80-9756-4e0e-a126-0444829c3435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:53.273740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-14086d80-9756-4e0e-a126-0444829c3435 None None] MFA Rules not processed for user `aefa27e6907c4ba28483d31a68753c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:53.275218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14086d80-9756-4e0e-a126-0444829c3435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:53.332589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:53.369904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 687/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Aug 18 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:08:53.378603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a7d65-a88f-4950-a011-0a6062b27243 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:53.380485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-505a7d65-a88f-4950-a011-0a6062b27243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:53.515513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a7d65-a88f-4950-a011-0a6062b27243 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:53.569719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:53.614811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:53.615342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:53.615744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:53.616172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:53.619292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:53.789228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:53.790783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8859af-712d-47dd-8397-4a26c06fe27b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:53.814406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 686/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:53.965250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f7059-70b8-46a3-b514-1123f991503e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:53.965776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f7059-70b8-46a3-b514-1123f991503e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:53.966893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f7059-70b8-46a3-b514-1123f991503e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:53.969971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 688/1374] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:53.975189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e86b-ca00-4c22-bea3-8d7c15684b19 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:53.975503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e86b-ca00-4c22-bea3-8d7c15684b19 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:53.975798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e86b-ca00-4c22-bea3-8d7c15684b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:54.078865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2197e86b-ca00-4c22-bea3-8d7c15684b19 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:54.080265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2197e86b-ca00-4c22-bea3-8d7c15684b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:54.116299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:54.221505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 687/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:54.516924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f7059-70b8-46a3-b514-1123f991503e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:54.518667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c96f7059-70b8-46a3-b514-1123f991503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:54.693887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f7059-70b8-46a3-b514-1123f991503e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:54.765072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:54.840715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:54.841410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:54.841692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:54.841978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:54.843763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:54.950745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:54.952678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8ecaa-ae77-420f-818f-70c87d1ebdf3 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:54.974247 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 689/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:08:57.091942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9053e5-ab26-4ef6-b0d0-fff2da441541 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:57.092561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9053e5-ab26-4ef6-b0d0-fff2da441541 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:57.092741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9053e5-ab26-4ef6-b0d0-fff2da441541 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:57.095016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 688/1377] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:08:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:08:57.101932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e3422-ada8-465b-b957-c895d7456691 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:57.102409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e3422-ada8-465b-b957-c895d7456691 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:57.102587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e3422-ada8-465b-b957-c895d7456691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:57.202212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bc9e3422-ada8-465b-b957-c895d7456691 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:08:57.203457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bc9e3422-ada8-465b-b957-c895d7456691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:57.210844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:57.356851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 690/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:08:57.663212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9053e5-ab26-4ef6-b0d0-fff2da441541 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:08:57.668018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a9053e5-ab26-4ef6-b0d0-fff2da441541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:57.895533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9053e5-ab26-4ef6-b0d0-fff2da441541 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:08:57.900435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:08:58.084361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:08:58.087194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:08:58.087194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:08:58.087194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:08:58.087194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:08:58.200929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:08:58.202644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e734-22ae-4db9-bbd9-ab43facf0625 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:08:58.223403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 689/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:00.348750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa82b-0863-46c1-9f58-a3ff6d59c0db None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:00.349328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa82b-0863-46c1-9f58-a3ff6d59c0db None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:00.349328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa82b-0863-46c1-9f58-a3ff6d59c0db None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:00.350838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 691/1380] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:00.357368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767ed6d-a269-4f0e-94f5-01a1d1d189cf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:00.357923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767ed6d-a269-4f0e-94f5-01a1d1d189cf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:00.357923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767ed6d-a269-4f0e-94f5-01a1d1d189cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:00.372652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:00.434467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c767ed6d-a269-4f0e-94f5-01a1d1d189cf None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:00.435716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c767ed6d-a269-4f0e-94f5-01a1d1d189cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:00.567333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 690/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:00.865007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fa82b-0863-46c1-9f58-a3ff6d59c0db None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:00.866818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2e1fa82b-0863-46c1-9f58-a3ff6d59c0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:01.034903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:01.049365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fa82b-0863-46c1-9f58-a3ff6d59c0db None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:01.186895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:01.187502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:01.187779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:01.188048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:01.189701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:01.300009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:01.301842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9ea90-4f6f-4a48-820e-be4d169b29f5 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:01.314903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:01.325510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 692/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:03.394335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea150204-c57a-4c7c-a62d-04491ee92b92 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:03.395023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea150204-c57a-4c7c-a62d-04491ee92b92 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:03.395023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea150204-c57a-4c7c-a62d-04491ee92b92 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:03.396134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 691/1383] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:03.402458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379286ac-3592-4444-aa9d-6b91dc4d27e7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:03.403500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379286ac-3592-4444-aa9d-6b91dc4d27e7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:03.403500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379286ac-3592-4444-aa9d-6b91dc4d27e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:03.481751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-379286ac-3592-4444-aa9d-6b91dc4d27e7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:03.482998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-379286ac-3592-4444-aa9d-6b91dc4d27e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:03.634379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 693/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:03.931627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea150204-c57a-4c7c-a62d-04491ee92b92 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:03.933387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ea150204-c57a-4c7c-a62d-04491ee92b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:04.048606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:04.125452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea150204-c57a-4c7c-a62d-04491ee92b92 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:04.270235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:04.271251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:04.271684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:04.272228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:04.274548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:04.352537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:04.409168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:04.411496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c148e6-68b9-4316-861f-3b0c85c75db0 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:04.439852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 692/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:06.633750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4300a4d-a22b-4359-b5db-5d0c029f9373 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:06.634421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4300a4d-a22b-4359-b5db-5d0c029f9373 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:06.634531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4300a4d-a22b-4359-b5db-5d0c029f9373 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:06.637083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 694/1386] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:06.645620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79c5a2-0a0f-48bb-9be2-8f5ca1678cdb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:06.646130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79c5a2-0a0f-48bb-9be2-8f5ca1678cdb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:06.646295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79c5a2-0a0f-48bb-9be2-8f5ca1678cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:06.720504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cf79c5a2-0a0f-48bb-9be2-8f5ca1678cdb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:06.721738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf79c5a2-0a0f-48bb-9be2-8f5ca1678cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:06.847392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 693/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:07.222349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4300a4d-a22b-4359-b5db-5d0c029f9373 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:07.223918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c4300a4d-a22b-4359-b5db-5d0c029f9373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:07.321420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:07.411340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4300a4d-a22b-4359-b5db-5d0c029f9373 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:07.552358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:07.552972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:07.553246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:07.553516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:07.555145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:07.599253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:07.662388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:07.664056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4a73c-096d-41ea-a52c-5c863ccbc890 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:07.684830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 695/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:09.645968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8d33f-62ec-43d8-b3b2-74daf9706ca6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:09.646564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8d33f-62ec-43d8-b3b2-74daf9706ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:09.646675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8d33f-62ec-43d8-b3b2-74daf9706ca6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:09.648002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 694/1389] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:09.654731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0a45-f7d2-4498-b9da-c4a097499293 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:09.655185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0a45-f7d2-4498-b9da-c4a097499293 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:09.655372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0a45-f7d2-4498-b9da-c4a097499293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:09.668639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61106910-da40-4d02-83d0-ee71eb6e6a2a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:09.668924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61106910-da40-4d02-83d0-ee71eb6e6a2a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:09.669198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61106910-da40-4d02-83d0-ee71eb6e6a2a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:09.670532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 695/1390] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:09:09.675582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b82b43-443c-4435-9b6e-19f13e2e4c81 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:09.675864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b82b43-443c-4435-9b6e-19f13e2e4c81 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:09.676134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b82b43-443c-4435-9b6e-19f13e2e4c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:09.712422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:09.725373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aada0a45-f7d2-4498-b9da-c4a097499293 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:09.726043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aada0a45-f7d2-4498-b9da-c4a097499293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:09.747092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-08b82b43-443c-4435-9b6e-19f13e2e4c81 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:09.748341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-08b82b43-443c-4435-9b6e-19f13e2e4c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:09.834132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 696/1391] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:09.853216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 696/1392] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:10.162712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd9c9-f8b6-4a56-9626-14b6dd2a85c5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:10.162996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd9c9-f8b6-4a56-9626-14b6dd2a85c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:10.163270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd9c9-f8b6-4a56-9626-14b6dd2a85c5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:10.164841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 697/1393] 10.222.0.16 () {54 vars in 967 bytes} [Thu Aug 18 11:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:10.170276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:10.170636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:10.170991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:10.193989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd9c9-f8b6-4a56-9626-14b6dd2a85c5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:10.195500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-163fd9c9-f8b6-4a56-9626-14b6dd2a85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:10.211359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:10.253066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] MFA Rules not processed for user `5e78415ea98940bbbc59c733c8ffefa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:10.254557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:10.306785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:10.337156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 697/1394] 10.222.0.16 () {58 vars in 1064 bytes} [Thu Aug 18 11:09:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:10.346217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:10.356873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f94a02-b208-4972-bec8-ade95128d249 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:10.367723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd9c9-f8b6-4a56-9626-14b6dd2a85c5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:10.443031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:10.443657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:10.443940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:10.444231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:10.446003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:10.476526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:10.515416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:10.516168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:10.516508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:10.517105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:10.518898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:10.594305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:10.595793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5343eb-244d-4be6-8b31-e0944e53b608 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:10.622262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:10.629818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 698/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:09:10.648303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:10.650007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d5b7d-daad-491f-9aaa-b03d17170cbd service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:10.672596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 698/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:09:11.430499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d8f6-aa5a-4ca1-9682-d84842bdfa8d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:11.430987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d8f6-aa5a-4ca1-9682-d84842bdfa8d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:11.431074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d8f6-aa5a-4ca1-9682-d84842bdfa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:11.503358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6a81d8f6-aa5a-4ca1-9682-d84842bdfa8d None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:11.504897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6a81d8f6-aa5a-4ca1-9682-d84842bdfa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:11.596340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 699/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:09:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:11.641033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4e661-cb88-4e5e-b753-82e52bb01a7c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:11.641316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4e661-cb88-4e5e-b753-82e52bb01a7c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:11.641593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4e661-cb88-4e5e-b753-82e52bb01a7c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:11.642998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 699/1398] 10.222.0.16 () {54 vars in 943 bytes} [Thu Aug 18 11:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:11.649654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404aa42-8a4b-4b54-844e-5b33e1ba2bd1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:11.649947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404aa42-8a4b-4b54-844e-5b33e1ba2bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:11.650221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404aa42-8a4b-4b54-844e-5b33e1ba2bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:11.735164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7404aa42-8a4b-4b54-844e-5b33e1ba2bd1 None None] MFA Rules not processed for user `37ca19cdbc204ba995b2d8d46a914fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:11.736521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7404aa42-8a4b-4b54-844e-5b33e1ba2bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:11.768737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:11.867133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 700/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Thu Aug 18 11:09:11 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:11.877005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4e661-cb88-4e5e-b753-82e52bb01a7c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:11.878526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-44a4e661-cb88-4e5e-b753-82e52bb01a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:11.986281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:12.041044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4e661-cb88-4e5e-b753-82e52bb01a7c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:12.154577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:12.155181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:12.155463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:12.155734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:12.157459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:12.262969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:12.264592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e56e9d-c787-4ade-9f48-88e3f526d45c service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:12.268203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:12.274575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 700/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:09:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:09:12.715166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fef60-c71b-447e-9d23-4023623bc705 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:12.715801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fef60-c71b-447e-9d23-4023623bc705 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:12.715801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fef60-c71b-447e-9d23-4023623bc705 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:12.717132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 701/1401] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:12.722965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65bed-05ff-4f4d-a053-a76f6a117165 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:12.723259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65bed-05ff-4f4d-a053-a76f6a117165 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:12.723542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65bed-05ff-4f4d-a053-a76f6a117165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:12.815481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-01e65bed-05ff-4f4d-a053-a76f6a117165 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:12.816822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01e65bed-05ff-4f4d-a053-a76f6a117165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:12.934504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 701/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:13.240634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fef60-c71b-447e-9d23-4023623bc705 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:13.242368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-987fef60-c71b-447e-9d23-4023623bc705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:13.478255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fef60-c71b-447e-9d23-4023623bc705 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:13.482564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:13.633030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:13.633553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:13.633835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:13.634106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:13.635764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:13.751516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:13.753320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd7b35-9c86-4e2e-b4d6-6f056732423e service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:13.776417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 702/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:15.818435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b51de-f0e6-4188-bd53-ef4c9ebd8ae9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:15.819017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b51de-f0e6-4188-bd53-ef4c9ebd8ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:15.819017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b51de-f0e6-4188-bd53-ef4c9ebd8ae9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:15.820517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 702/1404] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:15.826374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89bb0-2e06-4599-afa0-42f8e4dffdd7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:15.826906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89bb0-2e06-4599-afa0-42f8e4dffdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:15.826906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89bb0-2e06-4599-afa0-42f8e4dffdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:15.838674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:15.908968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a1f89bb0-2e06-4599-afa0-42f8e4dffdd7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:15.910592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1f89bb0-2e06-4599-afa0-42f8e4dffdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:16.051465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 703/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:16.389371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b51de-f0e6-4188-bd53-ef4c9ebd8ae9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:16.390651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fa1b51de-f0e6-4188-bd53-ef4c9ebd8ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:16.497315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:16.592158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b51de-f0e6-4188-bd53-ef4c9ebd8ae9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:16.759598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:16.760592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:16.761583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:16.762017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:16.766539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:16.816529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:16.874820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:16.876560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3d181-feaa-4a49-9ab5-34f4d62fb669 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:16.896820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 703/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:19.068305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f26582-e06b-4134-a414-4b7989887985 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:19.068871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f26582-e06b-4134-a414-4b7989887985 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:19.068871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f26582-e06b-4134-a414-4b7989887985 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:19.075531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 704/1407] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:09:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:19.081538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386ebd-d484-4e09-bbf2-badde3a0f35a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:19.082083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386ebd-d484-4e09-bbf2-badde3a0f35a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:19.082083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386ebd-d484-4e09-bbf2-badde3a0f35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:19.181029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-05386ebd-d484-4e09-bbf2-badde3a0f35a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:19.182288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-05386ebd-d484-4e09-bbf2-badde3a0f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:19.324502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 704/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:19.648720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f26582-e06b-4134-a414-4b7989887985 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:19.650280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e2f26582-e06b-4134-a414-4b7989887985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:19.771311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:19.850769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f26582-e06b-4134-a414-4b7989887985 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:20.003927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:20.003927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:20.003927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:20.003927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:20.004907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:20.071833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:20.124437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:20.126095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fa496-ead7-4916-afde-473838424ec4 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:20.147458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 705/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:29.321534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:30.144951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=24 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:35.394739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c14cd7-737c-4227-a70a-2807729a1645 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:35.395371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c14cd7-737c-4227-a70a-2807729a1645 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:35.395371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c14cd7-737c-4227-a70a-2807729a1645 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:35.396613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 705/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Aug 18 11:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:35.401677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2e4dc-5e14-4551-93c1-0a5f82ff96fc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:35.402205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2e4dc-5e14-4551-93c1-0a5f82ff96fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:35.402331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2e4dc-5e14-4551-93c1-0a5f82ff96fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:35.479419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-37f2e4dc-5e14-4551-93c1-0a5f82ff96fc None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:35.481277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37f2e4dc-5e14-4551-93c1-0a5f82ff96fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:35.588759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 706/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Aug 18 11:09:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:35.603021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f304e-49ca-4381-b25e-317666506e46 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:35.603316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f304e-49ca-4381-b25e-317666506e46 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:35.603594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f304e-49ca-4381-b25e-317666506e46 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:35.605643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 706/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu Aug 18 11:09:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:35.611164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553ef0e-1e65-4cc6-9990-0befe9ff10aa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:35.611449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553ef0e-1e65-4cc6-9990-0befe9ff10aa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:35.611731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553ef0e-1e65-4cc6-9990-0befe9ff10aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:35.691901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:35.694833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a553ef0e-1e65-4cc6-9990-0befe9ff10aa None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:35.696757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a553ef0e-1e65-4cc6-9990-0befe9ff10aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:35.845756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 707/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Aug 18 11:09:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:38.286298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a20df-76f1-43b8-a3af-e26697adbaba None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:38.286866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a20df-76f1-43b8-a3af-e26697adbaba None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:38.286866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a20df-76f1-43b8-a3af-e26697adbaba None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:38.288106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:38.294488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba62bba-cf17-4244-b741-fa98b31c343f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:38.294941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba62bba-cf17-4244-b741-fa98b31c343f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:38.295047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba62bba-cf17-4244-b741-fa98b31c343f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:38.362556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7ba62bba-cf17-4244-b741-fa98b31c343f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:38.363827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7ba62bba-cf17-4244-b741-fa98b31c343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:38.421670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:38.464440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:38.758396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a20df-76f1-43b8-a3af-e26697adbaba None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:38.760100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2b4a20df-76f1-43b8-a3af-e26697adbaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:38.915220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a20df-76f1-43b8-a3af-e26697adbaba None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:38.990879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:39.037649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:39.038260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:39.038539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:39.038804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:39.040467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:39.133524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:39.135125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a245819-37d6-47e8-be91-bf34fadd4092 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:39.154118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 708/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:40.064268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16089f-c51f-4144-8e08-28277fb75189 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:40.064833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16089f-c51f-4144-8e08-28277fb75189 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:40.064833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16089f-c51f-4144-8e08-28277fb75189 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:40.066173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:40.071002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f08738-296f-4edb-bc99-7421c6e43c67 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:40.071467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f08738-296f-4edb-bc99-7421c6e43c67 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:40.071575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f08738-296f-4edb-bc99-7421c6e43c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:40.150238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-08f08738-296f-4edb-bc99-7421c6e43c67 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:40.151460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-08f08738-296f-4edb-bc99-7421c6e43c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:40.178641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:40.256853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:40.413218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16089f-c51f-4144-8e08-28277fb75189 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:40.414751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd16089f-c51f-4144-8e08-28277fb75189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:40.564856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16089f-c51f-4144-8e08-28277fb75189 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:40.606745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:40.683774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:40.684340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:40.684628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:40.684912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:40.686664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:40.781152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:40.782820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a38bb-14f6-4b88-b4b2-ed4e14c95bc6 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:40.802465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 710/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:41.729448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5c68-903e-4227-93ec-8071a90722ac None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:41.730110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5c68-903e-4227-93ec-8071a90722ac None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:41.730110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5c68-903e-4227-93ec-8071a90722ac None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:41.731761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:41.737006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9772828-7ee2-40d0-af85-a5d41493923e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:41.737489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9772828-7ee2-40d0-af85-a5d41493923e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:41.737595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9772828-7ee2-40d0-af85-a5d41493923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:41.794292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:41.818459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c9772828-7ee2-40d0-af85-a5d41493923e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:41.819710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c9772828-7ee2-40d0-af85-a5d41493923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:41.923025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:42.079603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5c68-903e-4227-93ec-8071a90722ac None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:42.081170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c5a5c68-903e-4227-93ec-8071a90722ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:42.234571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5c68-903e-4227-93ec-8071a90722ac None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:42.248815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:42.355979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:42.356527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:42.356807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:42.357078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:42.358690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:42.452823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:42.454441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144ff44-a777-4b75-84c8-9e918cd5e034 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:42.473700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 711/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:09:43.356424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563ec8-a2f3-4b84-921c-0a1855600dad None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:43.357274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563ec8-a2f3-4b84-921c-0a1855600dad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:43.357274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563ec8-a2f3-4b84-921c-0a1855600dad None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:43.358406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:43.363820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650252f-69e0-4d05-982c-bf75794587e0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:43.364281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650252f-69e0-4d05-982c-bf75794587e0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:43.364463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650252f-69e0-4d05-982c-bf75794587e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:43.384642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:43.441102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5650252f-69e0-4d05-982c-bf75794587e0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:43.442310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5650252f-69e0-4d05-982c-bf75794587e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:43.544260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:43.695274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563ec8-a2f3-4b84-921c-0a1855600dad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:43.696814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f9563ec8-a2f3-4b84-921c-0a1855600dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:43.812132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:43.848301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563ec8-a2f3-4b84-921c-0a1855600dad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:43.962259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:43.962855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:43.963133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:43.963403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:43.965953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:44.049018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:44.062913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:44.064495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcd70b-ad9f-4381-9d58-eaf4836edac0 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:44.082473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 713/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:44.958858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f605053-1b04-4cdb-8e8b-52e14f093d80 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:44.959375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f605053-1b04-4cdb-8e8b-52e14f093d80 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:44.959482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f605053-1b04-4cdb-8e8b-52e14f093d80 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:44.961027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:44.966511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003890-f361-4a29-8173-124a8ac2a992 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:44.967009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003890-f361-4a29-8173-124a8ac2a992 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:44.967114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003890-f361-4a29-8173-124a8ac2a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:45.048251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a9003890-f361-4a29-8173-124a8ac2a992 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:45.049472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9003890-f361-4a29-8173-124a8ac2a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:45.154926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:45.307217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f605053-1b04-4cdb-8e8b-52e14f093d80 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:45.309429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0f605053-1b04-4cdb-8e8b-52e14f093d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:45.400049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:45.467938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f605053-1b04-4cdb-8e8b-52e14f093d80 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:45.583418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:45.583915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:45.584228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:45.584493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:45.586092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:45.637202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:45.679405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:45.681111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b182e-d4e2-42a6-b725-2e0866224207 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:45.699158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 714/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:46.519307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af3ea-bd02-40ca-9b32-b1d20542df83 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:46.519875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af3ea-bd02-40ca-9b32-b1d20542df83 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:46.519875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af3ea-bd02-40ca-9b32-b1d20542df83 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:46.521241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:46.526144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054463d1-0503-4959-8fce-0ce5d9374906 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:46.526556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054463d1-0503-4959-8fce-0ce5d9374906 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:46.527230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054463d1-0503-4959-8fce-0ce5d9374906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:46.606840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-054463d1-0503-4959-8fce-0ce5d9374906 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:46.608420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-054463d1-0503-4959-8fce-0ce5d9374906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:46.712552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:46.866700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af3ea-bd02-40ca-9b32-b1d20542df83 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:46.868149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d0af3ea-bd02-40ca-9b32-b1d20542df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:46.920603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:47.028253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af3ea-bd02-40ca-9b32-b1d20542df83 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:47.143097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:47.143611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:47.143887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:47.144155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:47.145780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:47.153734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:47.239755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:47.241395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d8389-33fe-4891-a166-d37b880f3c62 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:47.260599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 716/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:48.069031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c10a-68b7-4827-8d96-60933ca3e72b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:48.069613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c10a-68b7-4827-8d96-60933ca3e72b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:48.069613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c10a-68b7-4827-8d96-60933ca3e72b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:48.070935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:48.076439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261fbd-2a38-499d-9f02-7381e2a92cc5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:48.076899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261fbd-2a38-499d-9f02-7381e2a92cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:48.077020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261fbd-2a38-499d-9f02-7381e2a92cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:48.153479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2261fbd-2a38-499d-9f02-7381e2a92cc5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:48.154689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2261fbd-2a38-499d-9f02-7381e2a92cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:48.222956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:48.252931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:48.416253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c10a-68b7-4827-8d96-60933ca3e72b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:48.418511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-38a8c10a-68b7-4827-8d96-60933ca3e72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:48.437592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:48.574105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c10a-68b7-4827-8d96-60933ca3e72b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:48.663086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:48.696586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:48.697107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:48.697386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:48.697659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:48.699803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:48.795177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:48.796900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365fb48-1c4c-4a82-a6b1-eb22399617fb service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:48.815141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 717/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:49.637797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a7d54-8f30-4147-8ce3-d7c7b3d77da8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:49.638407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a7d54-8f30-4147-8ce3-d7c7b3d77da8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:49.638407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a7d54-8f30-4147-8ce3-d7c7b3d77da8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:49.639716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:49.644946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e733ed3-85ab-409c-bfe9-e0b9346c1cbf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:49.645421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e733ed3-85ab-409c-bfe9-e0b9346c1cbf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:49.645545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e733ed3-85ab-409c-bfe9-e0b9346c1cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:49.722640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3e733ed3-85ab-409c-bfe9-e0b9346c1cbf None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:49.724037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3e733ed3-85ab-409c-bfe9-e0b9346c1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:49.768119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:49.824853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:09:49.977489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a7d54-8f30-4147-8ce3-d7c7b3d77da8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:49.979003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-907a7d54-8f30-4147-8ce3-d7c7b3d77da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:50.132235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a7d54-8f30-4147-8ce3-d7c7b3d77da8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:50.186498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:50.247532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:50.248042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:50.248352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:50.248625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:50.250233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:50.359621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:50.361259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed279932-64cd-49e4-b0fa-15b0ed46242e service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:50.381347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 719/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:51.189282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa349a8-c132-4ea2-acf4-42650fc0b2ec None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:51.189858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa349a8-c132-4ea2-acf4-42650fc0b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:51.189858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa349a8-c132-4ea2-acf4-42650fc0b2ec None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:51.191192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:51.196132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50d7dc-914e-4b6c-b3ce-f462e9bf4a47 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:51.196610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50d7dc-914e-4b6c-b3ce-f462e9bf4a47 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:51.196709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50d7dc-914e-4b6c-b3ce-f462e9bf4a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:51.274384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ed50d7dc-914e-4b6c-b3ce-f462e9bf4a47 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:51.275618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed50d7dc-914e-4b6c-b3ce-f462e9bf4a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:51.278267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:51.377174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:51.530731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa349a8-c132-4ea2-acf4-42650fc0b2ec None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:51.532344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-efa349a8-c132-4ea2-acf4-42650fc0b2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:51.681046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa349a8-c132-4ea2-acf4-42650fc0b2ec None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:51.711707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:51.800663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:51.801276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:51.801554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:51.801825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:51.803436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:51.894480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:51.896078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c7c6a-b5d3-4e16-8216-f57dc1ce726b service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:51.913880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 720/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:53.769042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a0d99-6705-40dc-8cda-4fdfffd4c638 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:53.769616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a0d99-6705-40dc-8cda-4fdfffd4c638 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:53.769616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a0d99-6705-40dc-8cda-4fdfffd4c638 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:53.770998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:53.777463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f7daa-24a8-40e9-9824-59d29ddef27e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:53.777992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f7daa-24a8-40e9-9824-59d29ddef27e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:53.778128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f7daa-24a8-40e9-9824-59d29ddef27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:53.819577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:53.847449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f27f7daa-24a8-40e9-9824-59d29ddef27e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:53.848709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f27f7daa-24a8-40e9-9824-59d29ddef27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:53.961809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:54.257162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a0d99-6705-40dc-8cda-4fdfffd4c638 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:54.259801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-486a0d99-6705-40dc-8cda-4fdfffd4c638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:54.402967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a0d99-6705-40dc-8cda-4fdfffd4c638 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:54.463676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:54.535103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:54.535713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:54.535996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:54.536332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:54.537982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:54.675144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:54.676690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483c11b-4c77-41b6-93fd-87f48daa7df2 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:54.698295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 722/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:09:56.947568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6d729-f0a9-4535-a0c6-2c5f104e3cc8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:56.948174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6d729-f0a9-4535-a0c6-2c5f104e3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:56.948174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6d729-f0a9-4535-a0c6-2c5f104e3cc8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:56.950423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:09:56.956658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db4055-9fab-4b2e-a7fe-260a3f8836ce None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:56.957242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db4055-9fab-4b2e-a7fe-260a3f8836ce None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:56.957242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db4055-9fab-4b2e-a7fe-260a3f8836ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:57.036333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-72db4055-9fab-4b2e-a7fe-260a3f8836ce None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:09:57.037591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-72db4055-9fab-4b2e-a7fe-260a3f8836ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:57.052113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:57.148949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:09:57.437295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6d729-f0a9-4535-a0c6-2c5f104e3cc8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:09:57.438903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0cf6d729-f0a9-4535-a0c6-2c5f104e3cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:57.561213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6d729-f0a9-4535-a0c6-2c5f104e3cc8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:09:57.575210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:09:57.645296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:09:57.645902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:09:57.646177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:09:57.646446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:09:57.648065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:09:57.783859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:09:57.785333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff66847-c0e4-4e52-8274-2d802da2272d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:09:57.805339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 723/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:10:00.204332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24923fd-332b-4575-bc53-dbd9254c370e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:00.204926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24923fd-332b-4575-bc53-dbd9254c370e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:00.204926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24923fd-332b-4575-bc53-dbd9254c370e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:00.206269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:00.212068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e17a3-6012-411a-b096-74ac146ebd66 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:00.212620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e17a3-6012-411a-b096-74ac146ebd66 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:00.212734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e17a3-6012-411a-b096-74ac146ebd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:00.266545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:00.289998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a79e17a3-6012-411a-b096-74ac146ebd66 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:00.291127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a79e17a3-6012-411a-b096-74ac146ebd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:00.403647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:00.696787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24923fd-332b-4575-bc53-dbd9254c370e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:00.698461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f24923fd-332b-4575-bc53-dbd9254c370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:00.812473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24923fd-332b-4575-bc53-dbd9254c370e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:00.856560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:00.895287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:00.895887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:00.896170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:00.896480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:00.898181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:01.036735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:01.038188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8601bc-cfc0-49bb-aa87-125b6feb788a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:01.060038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 725/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:03.202266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c310e7-e96c-402d-8462-d81416057595 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:03.203108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c310e7-e96c-402d-8462-d81416057595 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:03.203108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c310e7-e96c-402d-8462-d81416057595 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:03.204588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:03.212043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff585-2821-491b-854a-8d56380f311e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:03.212575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff585-2821-491b-854a-8d56380f311e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:03.212680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff585-2821-491b-854a-8d56380f311e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:03.307960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-41bff585-2821-491b-854a-8d56380f311e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:03.309534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-41bff585-2821-491b-854a-8d56380f311e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:03.334130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:03.440475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:03.738278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c310e7-e96c-402d-8462-d81416057595 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:03.740664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-54c310e7-e96c-402d-8462-d81416057595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:03.891730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c310e7-e96c-402d-8462-d81416057595 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:03.913038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:03.980560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:03.981285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:03.981615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:03.981836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:03.983763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:04.106647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:04.108084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2013f-af4f-4e0c-a6b1-47e27505b85e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:04.126596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 726/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:06.172180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f148f-ab5e-47ce-91df-3a081a5c1494 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:06.172815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f148f-ab5e-47ce-91df-3a081a5c1494 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:06.172928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f148f-ab5e-47ce-91df-3a081a5c1494 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:06.174273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:06.180399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f20d1-5c9a-46cb-885f-ad2ab5f51762 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:06.180936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f20d1-5c9a-46cb-885f-ad2ab5f51762 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:06.181065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f20d1-5c9a-46cb-885f-ad2ab5f51762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:06.260936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:06.265976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a91f20d1-5c9a-46cb-885f-ad2ab5f51762 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:06.270650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a91f20d1-5c9a-46cb-885f-ad2ab5f51762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:06.394669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:06.697306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f148f-ab5e-47ce-91df-3a081a5c1494 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:06.699787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-973f148f-ab5e-47ce-91df-3a081a5c1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:06.822907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f148f-ab5e-47ce-91df-3a081a5c1494 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:06.877421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:06.903441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:06.904043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:06.904352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:06.905017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:06.906249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:07.026445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:07.027898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e75b80-d2ae-4401-9388-f5d4ca8f2fa1 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:07.047215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 728/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:09.071248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb5d4-3110-4166-96f2-70bea182cfc7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:09.071977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb5d4-3110-4166-96f2-70bea182cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:09.071977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb5d4-3110-4166-96f2-70bea182cfc7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:09.073162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:09.079492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5949-0911-4bb5-95e0-2359e165b90a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:09.080024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5949-0911-4bb5-95e0-2359e165b90a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:09.080119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5949-0911-4bb5-95e0-2359e165b90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:09.162302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a1bc5949-0911-4bb5-95e0-2359e165b90a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:09.163559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1bc5949-0911-4bb5-95e0-2359e165b90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:09.196507 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:09.274011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:10:09.563354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eb5d4-3110-4166-96f2-70bea182cfc7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:09.564953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e29eb5d4-3110-4166-96f2-70bea182cfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:09.677257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eb5d4-3110-4166-96f2-70bea182cfc7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:09.716783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:09.766461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:09.767064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:09.767339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:09.767609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:09.769303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:09.908758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:09.910195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f660d7-d16c-4c07-9437-95e05254da82 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:09.930163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 729/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:12.035761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da58c14-1a0b-4cf2-98c1-76df724319b3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:12.036334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da58c14-1a0b-4cf2-98c1-76df724319b3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:12.036334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da58c14-1a0b-4cf2-98c1-76df724319b3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:12.037697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:12.043473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab40e77-d113-4a64-a764-a8a2b94680d3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:12.044009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab40e77-d113-4a64-a764-a8a2b94680d3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:12.044009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab40e77-d113-4a64-a764-a8a2b94680d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:12.116173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fab40e77-d113-4a64-a764-a8a2b94680d3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:12.117428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fab40e77-d113-4a64-a764-a8a2b94680d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:12.126848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:12.222303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:12.513820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da58c14-1a0b-4cf2-98c1-76df724319b3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:12.515356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7da58c14-1a0b-4cf2-98c1-76df724319b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:12.625342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da58c14-1a0b-4cf2-98c1-76df724319b3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:12.682016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:12.701732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:12.702336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:12.702611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:12.702882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:12.704562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:12.818791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:12.820213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da999a-2488-4bbe-822c-881bdb5529fd service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:12.839874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 731/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:14.820402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366453a-2806-4d91-a942-a16c01686b88 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:14.820977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366453a-2806-4d91-a942-a16c01686b88 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:14.820977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366453a-2806-4d91-a942-a16c01686b88 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:14.822307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:14.828521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40313d17-88f3-4576-86ee-02b09d0c72d1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:14.829039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40313d17-88f3-4576-86ee-02b09d0c72d1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:14.829138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40313d17-88f3-4576-86ee-02b09d0c72d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:14.904937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-40313d17-88f3-4576-86ee-02b09d0c72d1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:14.906203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-40313d17-88f3-4576-86ee-02b09d0c72d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:14.954416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:15.028920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:15.353202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366453a-2806-4d91-a942-a16c01686b88 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:15.355292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0366453a-2806-4d91-a942-a16c01686b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:15.463379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366453a-2806-4d91-a942-a16c01686b88 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:15.503832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:15.544490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:15.545102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:15.545380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:15.545653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:15.547287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:15.685728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:15.688273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e32949-5bde-4a05-9113-38235494dd3d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:15.716987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 732/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:17.810916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ef19-23b6-4ecb-a96e-08be90e707cd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:17.811500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ef19-23b6-4ecb-a96e-08be90e707cd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:17.811500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512ef19-23b6-4ecb-a96e-08be90e707cd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:17.812860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:17.818349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f24e3f-e4f6-42cd-800c-fd9c81c6fd72 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:17.819126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f24e3f-e4f6-42cd-800c-fd9c81c6fd72 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:17.819126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f24e3f-e4f6-42cd-800c-fd9c81c6fd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:17.895219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-42f24e3f-e4f6-42cd-800c-fd9c81c6fd72 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:17.896487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-42f24e3f-e4f6-42cd-800c-fd9c81c6fd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:17.922444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:18.013632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:18.299673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512ef19-23b6-4ecb-a96e-08be90e707cd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:18.301263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c512ef19-23b6-4ecb-a96e-08be90e707cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:18.443313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512ef19-23b6-4ecb-a96e-08be90e707cd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:18.515784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:18.522971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:18.523579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:18.523858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:18.524131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:18.525781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:18.670325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:18.672892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdbabb-fe4c-4278-8491-026e6e807204 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:18.702176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 734/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:20.683475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44cb1-802a-4076-a3f2-ea4347fb435c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:20.684061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44cb1-802a-4076-a3f2-ea4347fb435c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:20.684061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44cb1-802a-4076-a3f2-ea4347fb435c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:20.685506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:20.694952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4d2c7-757d-4037-90f7-7e76fb048713 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:20.695500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4d2c7-757d-4037-90f7-7e76fb048713 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:20.695770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4d2c7-757d-4037-90f7-7e76fb048713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:20.795018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-98c4d2c7-757d-4037-90f7-7e76fb048713 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:20.796293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-98c4d2c7-757d-4037-90f7-7e76fb048713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:20.847653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:20.900800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:21.187716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca44cb1-802a-4076-a3f2-ea4347fb435c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:21.189284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0ca44cb1-802a-4076-a3f2-ea4347fb435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:21.361939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca44cb1-802a-4076-a3f2-ea4347fb435c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:21.414014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:21.453996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:21.454609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:21.454938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:21.455219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:21.457002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:21.582158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:21.585643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fed85d-733c-4c27-88eb-b1c8eadcf8d4 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:21.616269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 735/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:10:30.898848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=22 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:31.612899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=66 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:42.290629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1400d-8901-4aa0-bb2f-4a042bbaa59b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:42.291206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1400d-8901-4aa0-bb2f-4a042bbaa59b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:42.291299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1400d-8901-4aa0-bb2f-4a042bbaa59b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:42.293383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 736/1471] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:10:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:42.298974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfffa6-edd1-490c-9ae7-ff90f12abadb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:42.299505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfffa6-edd1-490c-9ae7-ff90f12abadb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:42.299505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfffa6-edd1-490c-9ae7-ff90f12abadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:42.505093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ebcfffa6-edd1-490c-9ae7-ff90f12abadb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:42.506310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ebcfffa6-edd1-490c-9ae7-ff90f12abadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:42.612293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 736/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:42.928874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c10482-8b40-4746-9d73-b9a4fcbe2b9e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:42.929155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c10482-8b40-4746-9d73-b9a4fcbe2b9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:42.929426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c10482-8b40-4746-9d73-b9a4fcbe2b9e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:42.930833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 737/1473] 10.222.0.16 () {54 vars in 967 bytes} [Thu Aug 18 11:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:42.938443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791a9b3-7d1b-4726-b5f8-d869b6716857 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:42.938916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791a9b3-7d1b-4726-b5f8-d869b6716857 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:42.939449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791a9b3-7d1b-4726-b5f8-d869b6716857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:43.033506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7791a9b3-7d1b-4726-b5f8-d869b6716857 None None] MFA Rules not processed for user `5e78415ea98940bbbc59c733c8ffefa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:43.035803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7791a9b3-7d1b-4726-b5f8-d869b6716857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:43.060165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:43.125363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 737/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Thu Aug 18 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:43.133391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c10482-8b40-4746-9d73-b9a4fcbe2b9e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:43.134938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03c10482-8b40-4746-9d73-b9a4fcbe2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:43.259844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c10482-8b40-4746-9d73-b9a4fcbe2b9e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:43.362726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:43.363461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:43.363587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:43.363853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:43.365487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:43.428467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:43.501351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:43.502784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fc25a-a8fb-4b05-8465-4801fbbdb0fe service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:43.528302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 738/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:45.649466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db137-1efc-4638-b706-00263e1521f1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:45.650026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db137-1efc-4638-b706-00263e1521f1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:45.650026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db137-1efc-4638-b706-00263e1521f1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:45.651394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 738/1476] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:45.657530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99757a-a172-411e-974c-417f3a762932 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:45.658037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99757a-a172-411e-974c-417f3a762932 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:45.658150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99757a-a172-411e-974c-417f3a762932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:45.725410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3f99757a-a172-411e-974c-417f3a762932 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:45.726623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f99757a-a172-411e-974c-417f3a762932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:45.830934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 739/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:46.112029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db137-1efc-4638-b706-00263e1521f1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:46.113606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9a3db137-1efc-4638-b706-00263e1521f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:46.243215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db137-1efc-4638-b706-00263e1521f1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:46.298650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:46.344055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:46.344964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:46.345239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:46.345745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:46.346508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:46.348253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:10:46.348253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:10:46.348253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:10:46.349383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcece5c2-6bd3-4d4a-ad3d-7b7d3d9aced0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:46.369202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 739/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Aug 18 11:10:46 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:10:48.349630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90468-ae42-44a8-946e-0e7083284764 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:48.350244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90468-ae42-44a8-946e-0e7083284764 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:48.350244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90468-ae42-44a8-946e-0e7083284764 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:48.351585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 740/1479] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:48.357675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8ef35-8f66-4761-bff9-78a979383d9c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:48.358105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8ef35-8f66-4761-bff9-78a979383d9c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:48.358214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8ef35-8f66-4761-bff9-78a979383d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:48.423318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-09a8ef35-8f66-4761-bff9-78a979383d9c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:48.424558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-09a8ef35-8f66-4761-bff9-78a979383d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:48.522307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 740/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:48.824449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e90468-ae42-44a8-946e-0e7083284764 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:48.825994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d0e90468-ae42-44a8-946e-0e7083284764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:48.839334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:48.988468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e90468-ae42-44a8-946e-0e7083284764 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:49.082148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:49.131517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:49.132417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:49.132699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:49.132971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:49.134589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:49.232417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:49.234079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a80a-9cf3-441b-a485-b174cfd1ec72 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:49.252450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 741/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:10:51.395093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c744b-b3a2-4d8f-ad7f-915d0e6aa53b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:51.395616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c744b-b3a2-4d8f-ad7f-915d0e6aa53b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:51.395616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c744b-b3a2-4d8f-ad7f-915d0e6aa53b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:51.397027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 741/1482] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:51.402536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae23de-75cb-42ff-bc3a-1808af1e3cbe None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:51.402977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae23de-75cb-42ff-bc3a-1808af1e3cbe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:51.403084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae23de-75cb-42ff-bc3a-1808af1e3cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:51.474087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-90ae23de-75cb-42ff-bc3a-1808af1e3cbe None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:51.475311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-90ae23de-75cb-42ff-bc3a-1808af1e3cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:51.523914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:51.580165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 742/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:51.868251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c744b-b3a2-4d8f-ad7f-915d0e6aa53b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:51.870563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9d6c744b-b3a2-4d8f-ad7f-915d0e6aa53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:51.896453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:51.984957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c744b-b3a2-4d8f-ad7f-915d0e6aa53b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:52.066674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:52.067200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:52.067479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:52.067754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:52.069415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:52.149447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:52.188430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:52.189872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd755d43-3023-450b-9fc8-a5c09d512e08 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:52.209624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 742/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:54.219750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1057-7ab5-4210-96ff-d0363675c1d0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:54.220353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1057-7ab5-4210-96ff-d0363675c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:54.220353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1057-7ab5-4210-96ff-d0363675c1d0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:54.221708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 743/1485] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:54.227333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1364e-9a5f-4434-83cd-3606ad5b317e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:54.227860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1364e-9a5f-4434-83cd-3606ad5b317e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:54.227860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1364e-9a5f-4434-83cd-3606ad5b317e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:54.298118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4ac1364e-9a5f-4434-83cd-3606ad5b317e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:54.299249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4ac1364e-9a5f-4434-83cd-3606ad5b317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:54.406317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 743/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:54.695245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1057-7ab5-4210-96ff-d0363675c1d0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:54.696798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-760b1057-7ab5-4210-96ff-d0363675c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:54.807468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1057-7ab5-4210-96ff-d0363675c1d0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:54.889189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:54.891689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:54.892562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:54.893062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:54.893525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:54.895301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:55.144203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:55.145648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c5166-a447-4f96-94f0-e2462ae815a6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:55.164361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 744/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:10:57.268927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27a52e-c39a-417b-add0-3f07712d694b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:57.269511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27a52e-c39a-417b-add0-3f07712d694b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:57.269511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27a52e-c39a-417b-add0-3f07712d694b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:57.270847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 744/1488] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:10:57.276754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392f945-fe5e-470a-86fb-31c216846674 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:57.277313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392f945-fe5e-470a-86fb-31c216846674 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:57.277456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392f945-fe5e-470a-86fb-31c216846674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:57.354226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6392f945-fe5e-470a-86fb-31c216846674 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:10:57.355960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6392f945-fe5e-470a-86fb-31c216846674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:57.414313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:57.461400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 745/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:10:57.760703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27a52e-c39a-417b-add0-3f07712d694b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:10:57.762401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5e27a52e-c39a-417b-add0-3f07712d694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:57.799415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:57.869088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27a52e-c39a-417b-add0-3f07712d694b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:10:57.944059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:10:57.944690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:10:57.944969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:10:57.945228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:10:57.946818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:10:58.030073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:10:58.060871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:10:58.062305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb536bbf-ba92-4cfb-8d77-3efe568454ad service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:10:58.082815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 745/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:11:00.262729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4852478-9de1-43b3-88d4-f67e1fe877f4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:11:00.263316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4852478-9de1-43b3-88d4-f67e1fe877f4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:11:00.263316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4852478-9de1-43b3-88d4-f67e1fe877f4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:11:00.264656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 746/1491] 10.222.0.16 () {54 vars in 927 bytes} [Thu Aug 18 11:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:11:00.270206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440b73b-f49d-4fc3-aefc-278a64f5fc52 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:11:00.270647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440b73b-f49d-4fc3-aefc-278a64f5fc52 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:11:00.270759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440b73b-f49d-4fc3-aefc-278a64f5fc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:11:00.337138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e440b73b-f49d-4fc3-aefc-278a64f5fc52 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:11:00.338366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e440b73b-f49d-4fc3-aefc-278a64f5fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:11:00.436611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 746/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Aug 18 11:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:11:00.725134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4852478-9de1-43b3-88d4-f67e1fe877f4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:11:00.726676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4852478-9de1-43b3-88d4-f67e1fe877f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:11:00.832506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4852478-9de1-43b3-88d4-f67e1fe877f4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:11:00.908068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:11:00.908693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:11:00.908978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:11:00.909245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:11:00.910858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:11:00.920128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:11:01.023857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:11:01.025263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5373fd-a595-43c7-9a42-803b8f27b607 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:11:01.044369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 747/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:11:10.434691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=82 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:11:11.041444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=50 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:21.053260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:21.053834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:21.053834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:21.128492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:12:21.129709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:21.224505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 747/1494] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:12:21.232646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:21.233182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:21.233182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:21.309140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:12:21.310285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:21.406626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 748/1495] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:12:21.411325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:21.412542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:21.424105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e9f3d-35b4-4758-a3a8-e96f90c6ce59 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:21.485908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:21.530902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:21.531512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:21.531789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:21.532051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:21.532810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:12:21.533107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:21.534108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:21.534108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:12:21.534108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:12:21.534995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:21.547160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 748/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:12:21.555353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:21.565246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79266643-8d2d-435d-8747-52f4140064c6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:21.627350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:21.661448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:21.662043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:21.662318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:21.662580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:21.663212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:21.665058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:21.665058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:12:21.665058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:12:21.666234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:21.722532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 749/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:12:21 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:12:21.726993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:21.728601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:21.763453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ba73-6e34-4d6b-a706-2f48a61b0312 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:21.857933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:21.858538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:21.858810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:21.859072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:21.859773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:21.860715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:21.860715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:12:21.860715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:12:21.861915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:21.905321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 749/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:12:21 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:12:21.909638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:21.911177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:21.972979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c642c8-9659-4481-aa55-c9aa04db549f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:22.062524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:22.074543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:22.075145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.075423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.075695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.076378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:22.077393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:22.077393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:12:22.077393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:12:22.078279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:22.091310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 750/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:12:22.095412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:22.096990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:22.109309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e81a6-0b4f-402a-acdb-d24c2f2da31a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:22.135283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:22.208082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:22.208717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.208993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.209260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.209888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:22.210886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:22.210886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:12:22.210886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:12:22.211763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808d62-805f-428c-82c0-1861b5cb18e8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:22.224527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 750/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:12:22.228711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:22.230972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:22.264183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116435aa-deea-4404-9b52-fd86cdc94bdc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:22.359583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:22.360161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.360469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.360738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] PATH_INFO: `/v3/projects/5d7861c1049b49ee83dd75f55b3ad51e/users/436629382640410586c73a7ba752a9bd/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.382510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7861c1049b49ee83dd75f55b3ad51e, user_id=436629382640410586c73a7ba752a9bd, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:22.383540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:22.383540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:12:22.383540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:12:22.386348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8fdb96-509c-41ca-bb77-5096d6331e23 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:22.410840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 751/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:12:22 2022] PUT /identity/v3/projects/5d7861c1049b49ee83dd75f55b3ad51e/users/436629382640410586c73a7ba752a9bd/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:12:22.423704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d17b4c-3d70-45fa-a599-fb4f465b75f8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.423991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d17b4c-3d70-45fa-a599-fb4f465b75f8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.424300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d17b4c-3d70-45fa-a599-fb4f465b75f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.481334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b9d17b4c-3d70-45fa-a599-fb4f465b75f8 None None] MFA Rules not processed for user `436629382640410586c73a7ba752a9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:12:22.482544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b9d17b4c-3d70-45fa-a599-fb4f465b75f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:22.561230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 751/1502] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:12:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:12:22.574699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af258-4382-4a28-9fed-117ca157fa94 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.574981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af258-4382-4a28-9fed-117ca157fa94 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.575255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af258-4382-4a28-9fed-117ca157fa94 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.576688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 752/1503] 10.222.0.16 () {54 vars in 964 bytes} [Thu Aug 18 11:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:12:22.581658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e880a-edff-4240-b30e-c737d73cacf8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.581947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e880a-edff-4240-b30e-c737d73cacf8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.582225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e880a-edff-4240-b30e-c737d73cacf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.613777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:22.659846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cd7e880a-edff-4240-b30e-c737d73cacf8 None None] MFA Rules not processed for user `aefa27e6907c4ba28483d31a68753c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:12:22.661113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cd7e880a-edff-4240-b30e-c737d73cacf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:22.736372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 752/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Aug 18 11:12:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:12:22.744261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98af258-4382-4a28-9fed-117ca157fa94 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:22.746438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a98af258-4382-4a28-9fed-117ca157fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:22.796892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:22.852896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98af258-4382-4a28-9fed-117ca157fa94 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:22.927020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:22.927621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:22.927903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:22.928177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:22.929823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:23.029868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:23.031315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:23.034861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:23.050569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 753/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:12:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:12:23.095346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c957c7b-3f01-45f8-8b4e-b19795791fae None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:23.095634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c957c7b-3f01-45f8-8b4e-b19795791fae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:23.096115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c957c7b-3f01-45f8-8b4e-b19795791fae None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:23.097198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 753/1506] 10.222.0.16 () {52 vars in 830 bytes} [Thu Aug 18 11:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:12:23.100864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:23.102067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:23.203885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e82ac-930b-456c-b416-84eeab8c6ef7 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:23.298162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:23.298885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:23.299052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:23.299321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] PATH_INFO: `/v3/users/436629382640410586c73a7ba752a9bd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:23.312249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436629382640410586c73a7ba752a9bd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:23.313267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:23.313267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:12:23.313267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:12:23.314479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:23.324382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:23.380461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436629382640410586c73a7ba752a9bd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:23.380998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436629382640410586c73a7ba752a9bd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:23.381423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436629382640410586c73a7ba752a9bd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:23.405046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436629382640410586c73a7ba752a9bd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:23.415661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436629382640410586c73a7ba752a9bd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:23.435945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 754/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:12:23 2022] DELETE /identity/v3/users/436629382640410586c73a7ba752a9bd => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:12:23.440353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:23.440644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:23.440947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:23.498869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:12:23.500332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:23.579258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:23.593559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 754/1508] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:12:23.603084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:23.605236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:23.704182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cb5bc-6386-48c4-a9b9-c779d87ebe05 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:23.761884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:23.782809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:23.783405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:23.783704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:23.783975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:23.785575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:23.898547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:23.899977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863549-a7d1-44e7-8293-5eaab8c51539 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:23.919853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 755/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:12:24.251620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:12:24.252899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:12:24.266512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0320b-9a41-48f3-bb24-bd01c068c56b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:12:24.366637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:12:24.367293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:12:24.367566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:12:24.367832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] PATH_INFO: `/v3/projects/5d7861c1049b49ee83dd75f55b3ad51e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:12:24.378739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7861c1049b49ee83dd75f55b3ad51e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:12:24.379756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:12:24.379756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:12:24.379756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:12:24.380982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:12:24.439598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7861c1049b49ee83dd75f55b3ad51e', 'request_id': 'req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:24.445517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7861c1049b49ee83dd75f55b3ad51e', 'request_id': 'req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:24.455533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7861c1049b49ee83dd75f55b3ad51e', 'request_id': 'req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:24.486583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] The token cache is being invalidate because project 5d7861c1049b49ee83dd75f55b3ad51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:12:24.486861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:24.487279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-442be8bc-7abc-4e8e-88f8-cfb2fea4f9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:12:24.488681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 755/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:12:24 2022] DELETE /identity/v3/projects/5d7861c1049b49ee83dd75f55b3ad51e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:12:33.917692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=62 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:12:34.484570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:13:58.486932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccdbb5-c011-4076-ac0e-5651891b5caa None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:13:58.487555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccdbb5-c011-4076-ac0e-5651891b5caa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:13:58.487555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccdbb5-c011-4076-ac0e-5651891b5caa None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:13:58.488855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:13:58.493774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27c984-52f2-42f8-a913-8609b2cb3509 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:13:58.494194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27c984-52f2-42f8-a913-8609b2cb3509 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:13:58.494337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27c984-52f2-42f8-a913-8609b2cb3509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:13:58.583034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5d27c984-52f2-42f8-a913-8609b2cb3509 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:13:58.584307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5d27c984-52f2-42f8-a913-8609b2cb3509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:13:58.697412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:13:58.853280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccdbb5-c011-4076-ac0e-5651891b5caa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:13:58.855363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-deccdbb5-c011-4076-ac0e-5651891b5caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:13:58.974827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccdbb5-c011-4076-ac0e-5651891b5caa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:13:58.980344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bb49d-37f0-4796-9763-4b407dcc89d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:13:58.980961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb49d-37f0-4796-9763-4b407dcc89d4 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:13:58.981242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb49d-37f0-4796-9763-4b407dcc89d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:13:58.981704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb49d-37f0-4796-9763-4b407dcc89d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:13:58.992697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bb49d-37f0-4796-9763-4b407dcc89d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:13:58.994299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bb49d-37f0-4796-9763-4b407dcc89d4 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:13:59.004382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 757/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:13:59.861802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413e05-965c-44c4-bd8e-e07b35ea720c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:13:59.862379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413e05-965c-44c4-bd8e-e07b35ea720c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:13:59.862379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413e05-965c-44c4-bd8e-e07b35ea720c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:13:59.863739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:13:59.868659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f08f4-1361-4497-af1e-fe0ffb3f2437 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:13:59.869107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f08f4-1361-4497-af1e-fe0ffb3f2437 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:13:59.869218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f08f4-1361-4497-af1e-fe0ffb3f2437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:13:59.889774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0d1f08f4-1361-4497-af1e-fe0ffb3f2437 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:13:59.891042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d1f08f4-1361-4497-af1e-fe0ffb3f2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:13:59.899090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:00.053290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31413e05-965c-44c4-bd8e-e07b35ea720c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:00.067804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31413e05-965c-44c4-bd8e-e07b35ea720c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:00.073241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eef906-36be-48b7-bf36-ecb17486cf9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:00.073845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef906-36be-48b7-bf36-ecb17486cf9c service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:00.074214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef906-36be-48b7-bf36-ecb17486cf9c service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:00.074399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef906-36be-48b7-bf36-ecb17486cf9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:00.083099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eef906-36be-48b7-bf36-ecb17486cf9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:00.084762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eef906-36be-48b7-bf36-ecb17486cf9c service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:00.094911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 758/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:14:01.068108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23ce1-3d11-4c0e-bebb-cebccfb5154a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:01.068727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23ce1-3d11-4c0e-bebb-cebccfb5154a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:01.068727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23ce1-3d11-4c0e-bebb-cebccfb5154a None None] PATH_INFO: `/v3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:01.069957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 759/1517] 10.222.0.20 () {54 vars in 925 bytes} [Thu Aug 18 11:14:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:14:01.081754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ba275-ddb4-43d6-adef-3b56485bec7d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:01.082304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ba275-ddb4-43d6-adef-3b56485bec7d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:01.082304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ba275-ddb4-43d6-adef-3b56485bec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:01.145842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-455ba275-ddb4-43d6-adef-3b56485bec7d None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:01.147068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-455ba275-ddb4-43d6-adef-3b56485bec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:01.165417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:14:01.223853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 759/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Aug 18 11:14:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:05.412956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dd0a2-e431-4ed1-88bc-c8643e1dbffd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:05.413882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dd0a2-e431-4ed1-88bc-c8643e1dbffd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:05.413882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dd0a2-e431-4ed1-88bc-c8643e1dbffd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:05.415383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 760/1519] 10.222.0.16 () {54 vars in 961 bytes} [Thu Aug 18 11:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:05.421861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd86f03-df87-4523-a24b-b9b749914c73 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:05.422444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd86f03-df87-4523-a24b-b9b749914c73 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:05.422444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd86f03-df87-4523-a24b-b9b749914c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:05.443985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8bd86f03-df87-4523-a24b-b9b749914c73 None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:05.445282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8bd86f03-df87-4523-a24b-b9b749914c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:05.454413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 760/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Aug 18 11:14:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:14:05.462257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dd0a2-e431-4ed1-88bc-c8643e1dbffd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:05.477344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dd0a2-e431-4ed1-88bc-c8643e1dbffd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:05.483420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e8911-43aa-49bc-828f-2adb4e0bf623 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:05.484038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8911-43aa-49bc-828f-2adb4e0bf623 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:05.484355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8911-43aa-49bc-828f-2adb4e0bf623 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:05.484621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8911-43aa-49bc-828f-2adb4e0bf623 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:05.493850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e8911-43aa-49bc-828f-2adb4e0bf623 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:05.495455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e8911-43aa-49bc-828f-2adb4e0bf623 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:05.506690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 761/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:14:11.365831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a8a4e-0657-49fa-8324-bca03198d47c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:11.366405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a8a4e-0657-49fa-8324-bca03198d47c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:11.366405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a8a4e-0657-49fa-8324-bca03198d47c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:11.367727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:11.373965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07b7da-60fb-421a-941a-b2a8ff155279 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:11.374552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07b7da-60fb-421a-941a-b2a8ff155279 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:11.374552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07b7da-60fb-421a-941a-b2a8ff155279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:11.396484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8c07b7da-60fb-421a-941a-b2a8ff155279 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:11.397743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8c07b7da-60fb-421a-941a-b2a8ff155279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:11.406527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:11.571179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a8a4e-0657-49fa-8324-bca03198d47c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:11.589814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a8a4e-0657-49fa-8324-bca03198d47c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:11.596612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbab5d-0ced-44f8-a99c-4876550ce645 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:11.597335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbab5d-0ced-44f8-a99c-4876550ce645 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:11.597718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbab5d-0ced-44f8-a99c-4876550ce645 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:11.598036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbab5d-0ced-44f8-a99c-4876550ce645 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:11.609293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbab5d-0ced-44f8-a99c-4876550ce645 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:11.611238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbab5d-0ced-44f8-a99c-4876550ce645 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:11.623495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 762/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:14:12.517804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567b07-a728-440e-a641-ccd8cbe45757 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:12.518406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567b07-a728-440e-a641-ccd8cbe45757 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:12.518406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567b07-a728-440e-a641-ccd8cbe45757 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:12.519775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 11:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:12.524989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8af091-096d-45f0-9f6a-9660fbf1a40e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:12.525460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8af091-096d-45f0-9f6a-9660fbf1a40e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:12.525608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8af091-096d-45f0-9f6a-9660fbf1a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:12.546285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8b8af091-096d-45f0-9f6a-9660fbf1a40e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:12.547507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8b8af091-096d-45f0-9f6a-9660fbf1a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:12.555718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:12.726885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567b07-a728-440e-a641-ccd8cbe45757 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:12.741850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567b07-a728-440e-a641-ccd8cbe45757 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:12.748839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1286903-1401-4b6b-a1f5-e7a5d581e087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:12.749447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1286903-1401-4b6b-a1f5-e7a5d581e087 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:12.749733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1286903-1401-4b6b-a1f5-e7a5d581e087 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:12.750008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1286903-1401-4b6b-a1f5-e7a5d581e087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:12.759315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1286903-1401-4b6b-a1f5-e7a5d581e087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:12.760977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1286903-1401-4b6b-a1f5-e7a5d581e087 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:12.772984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 764/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:14:17.050820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3a99-627b-49c9-9df2-30d8c788fcba None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:17.051384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3a99-627b-49c9-9df2-30d8c788fcba None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:17.051384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3a99-627b-49c9-9df2-30d8c788fcba None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:17.052785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:17.058096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819c44-ae97-43c4-a2d0-f9142467bd3c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:17.058688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819c44-ae97-43c4-a2d0-f9142467bd3c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:17.058688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819c44-ae97-43c4-a2d0-f9142467bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:17.079370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d4819c44-ae97-43c4-a2d0-f9142467bd3c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:17.080596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d4819c44-ae97-43c4-a2d0-f9142467bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:17.088754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:17.384380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb3a99-627b-49c9-9df2-30d8c788fcba None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:17.386058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-70fb3a99-627b-49c9-9df2-30d8c788fcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:17.470058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb3a99-627b-49c9-9df2-30d8c788fcba None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:17.474245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62251a4-f8bf-40b3-9641-bf6ccd1f05a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:17.474858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62251a4-f8bf-40b3-9641-bf6ccd1f05a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:17.475138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62251a4-f8bf-40b3-9641-bf6ccd1f05a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:17.475411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62251a4-f8bf-40b3-9641-bf6ccd1f05a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:17.485361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62251a4-f8bf-40b3-9641-bf6ccd1f05a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:17.486795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62251a4-f8bf-40b3-9641-bf6ccd1f05a7 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:17.496723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 765/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:14:19.434015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0c44-7a34-45f2-94bc-c3c3a7e2adbf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:19.434598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0c44-7a34-45f2-94bc-c3c3a7e2adbf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:19.434598 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0c44-7a34-45f2-94bc-c3c3a7e2adbf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:19.435924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:19.441600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6ec8c-28ec-4a2e-86a2-f729071ffb9a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:19.442062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6ec8c-28ec-4a2e-86a2-f729071ffb9a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:19.442169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6ec8c-28ec-4a2e-86a2-f729071ffb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:19.463779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f9a6ec8c-28ec-4a2e-86a2-f729071ffb9a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:19.465039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f9a6ec8c-28ec-4a2e-86a2-f729071ffb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:19.474082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:19.761779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b0c44-7a34-45f2-94bc-c3c3a7e2adbf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:19.772866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b0c44-7a34-45f2-94bc-c3c3a7e2adbf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:19.776376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0238639-77fa-4b2f-b59e-d9a50498bff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:19.776972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0238639-77fa-4b2f-b59e-d9a50498bff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:19.777250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0238639-77fa-4b2f-b59e-d9a50498bff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:19.777520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0238639-77fa-4b2f-b59e-d9a50498bff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:19.787504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0238639-77fa-4b2f-b59e-d9a50498bff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:19.788969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0238639-77fa-4b2f-b59e-d9a50498bff9 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:19.799064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 767/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:14:23.454679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f06b-ce21-40a1-bc62-f25d802c21fe None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:23.455317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f06b-ce21-40a1-bc62-f25d802c21fe None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:23.455317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f06b-ce21-40a1-bc62-f25d802c21fe None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:23.456713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:23.462264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7fa42-1386-4e6d-9390-4f3e6e289cb5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:23.462725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7fa42-1386-4e6d-9390-4f3e6e289cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:23.462825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7fa42-1386-4e6d-9390-4f3e6e289cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:23.483531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d1c7fa42-1386-4e6d-9390-4f3e6e289cb5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:23.484776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d1c7fa42-1386-4e6d-9390-4f3e6e289cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:23.492719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:24.055236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584f06b-ce21-40a1-bc62-f25d802c21fe None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:24.056966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4584f06b-ce21-40a1-bc62-f25d802c21fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:24.126873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:14:24.142083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584f06b-ce21-40a1-bc62-f25d802c21fe None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:24.145767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561dec7-087d-490b-8fdc-152cc739448b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:24.146372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561dec7-087d-490b-8fdc-152cc739448b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:24.146650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561dec7-087d-490b-8fdc-152cc739448b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:24.146922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561dec7-087d-490b-8fdc-152cc739448b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:24.156447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561dec7-087d-490b-8fdc-152cc739448b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:24.157887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561dec7-087d-490b-8fdc-152cc739448b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:24.167465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 768/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:14:26.118718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837784-d041-4fff-9add-a4b15a321ca4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:26.119314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837784-d041-4fff-9add-a4b15a321ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:26.119314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837784-d041-4fff-9add-a4b15a321ca4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:26.120634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 11:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:14:26.126822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e568c83-5531-4aea-9f44-751ec6d72fd7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:26.127313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e568c83-5531-4aea-9f44-751ec6d72fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:26.127417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e568c83-5531-4aea-9f44-751ec6d72fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:26.148117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8e568c83-5531-4aea-9f44-751ec6d72fd7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:14:26.149378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8e568c83-5531-4aea-9f44-751ec6d72fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:26.157676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:14:26.722112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837784-d041-4fff-9add-a4b15a321ca4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:14:26.723784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d5837784-d041-4fff-9add-a4b15a321ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:14:26.731309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:14:26.737941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837784-d041-4fff-9add-a4b15a321ca4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:14:26.741470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981b88e-0817-4059-b6b1-4a71ac05c190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:14:26.742067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981b88e-0817-4059-b6b1-4a71ac05c190 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:14:26.742347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981b88e-0817-4059-b6b1-4a71ac05c190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:14:26.742663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981b88e-0817-4059-b6b1-4a71ac05c190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:14:26.751965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981b88e-0817-4059-b6b1-4a71ac05c190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:14:26.753438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981b88e-0817-4059-b6b1-4a71ac05c190 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:14:26.763118 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 770/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:14:36.143089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=11 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:14:36.758669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:52.405749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180d96b-e9db-432e-a1b3-9cf87fc703f7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.406870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180d96b-e9db-432e-a1b3-9cf87fc703f7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.407000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180d96b-e9db-432e-a1b3-9cf87fc703f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.438103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3321d-e2a1-4ed2-9dcc-10ec27e1ced9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.438728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3321d-e2a1-4ed2-9dcc-10ec27e1ced9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.439028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3321d-e2a1-4ed2-9dcc-10ec27e1ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.441328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5180d96b-e9db-432e-a1b3-9cf87fc703f7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.442570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5180d96b-e9db-432e-a1b3-9cf87fc703f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:52.459194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 770/1540] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:52.475463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cdd3321d-e2a1-4ed2-9dcc-10ec27e1ced9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.476760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cdd3321d-e2a1-4ed2-9dcc-10ec27e1ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:52.483791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.484296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.484835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.495498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 771/1541] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:52.509219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.509805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.510250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.521569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.523110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:52.542851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 771/1542] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:15:52.549988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.552063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:52.553260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:52.566700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 772/1543] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:15:52.567389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa054cb3-61ac-42c9-88d1-887b017ada43 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:52.571728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:52.588285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0db0f-d449-4389-9335-e0813e21ea4e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:52.593138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:52.594141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.594141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.594393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.594970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:52.595274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:52.598883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:52.601288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:52.602204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.602770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.603216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.604048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:52.605020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:52.606285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:15:52.606285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:15:52.606285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:15:52.607625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:52.622981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 772/1544] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:52.640713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 773/1545] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:15:52.649566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:52.649806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:52.673490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd903dc-fae5-4fca-a947-cf084a8f272f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:52.680229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c501-b53a-4cf4-80b9-3465f21b2e0c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:52.680964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9f368-af85-413b-bec4-45a1832a5ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:52.681605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f368-af85-413b-bec4-45a1832a5ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.681891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f368-af85-413b-bec4-45a1832a5ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.682180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f368-af85-413b-bec4-45a1832a5ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.683499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9f368-af85-413b-bec4-45a1832a5ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:52.687808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9f368-af85-413b-bec4-45a1832a5ef5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:52.701075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:52.702504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.702797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.703072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.703705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:52.704756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:15:52.704756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:15:52.704756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:15:52.705968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad410f3f-0d8a-48e7-9aee-b4194a379007 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:52.782481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 774/1546] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:52.783996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.784403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.784692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.801872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 773/1547] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/projects => generated 393 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:52.803188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.803476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.803750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.833413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.834676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:52.845464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.846713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:52.923206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 775/1548] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:52.924129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:52.935654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f35ea-daa6-486c-ba26-71a1786cc22f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:52.941903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:52.942590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.942871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.943144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.943776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:52.944886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:15:52.944886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:15:52.944886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:15:52.946408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066d07c-e177-4272-a1c8-e509e78e97fe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:52.973343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 774/1549] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:52.973759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 776/1550] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:52.974159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:52.975081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76461c8c-db19-46e3-9cbd-b1452a774eef None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.975368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76461c8c-db19-46e3-9cbd-b1452a774eef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.975640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76461c8c-db19-46e3-9cbd-b1452a774eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.985033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e20128-890e-4cff-9504-bd9085fdc262 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:52.989952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a2337-82d3-4c83-a3d0-405f52930039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:52.990571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a2337-82d3-4c83-a3d0-405f52930039 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:52.990853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a2337-82d3-4c83-a3d0-405f52930039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:52.991128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a2337-82d3-4c83-a3d0-405f52930039 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:52.991796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a2337-82d3-4c83-a3d0-405f52930039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:52.993768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a2337-82d3-4c83-a3d0-405f52930039 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:52.996880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-76461c8c-db19-46e3-9cbd-b1452a774eef None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:52.998762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-76461c8c-db19-46e3-9cbd-b1452a774eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.007184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 777/1551] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.009454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25513c48-bed9-43a4-8711-89b305638666 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.009736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25513c48-bed9-43a4-8711-89b305638666 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.010011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25513c48-bed9-43a4-8711-89b305638666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.027964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 775/1552] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:52 2022] POST /identity/v3/users => generated 297 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.027964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94011a58-5bc7-4ff4-a630-07126a9034f1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.027964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94011a58-5bc7-4ff4-a630-07126a9034f1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.028465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94011a58-5bc7-4ff4-a630-07126a9034f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.038678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-25513c48-bed9-43a4-8711-89b305638666 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.040842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-25513c48-bed9-43a4-8711-89b305638666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.048637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-94011a58-5bc7-4ff4-a630-07126a9034f1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.049842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-94011a58-5bc7-4ff4-a630-07126a9034f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.051589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 778/1553] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.052951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494681b3-dc05-4a03-919b-f71450355267 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.053230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494681b3-dc05-4a03-919b-f71450355267 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.053502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494681b3-dc05-4a03-919b-f71450355267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.067726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 776/1554] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.067726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079c1f4-245d-4723-9c15-1e6154777cd6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.067726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079c1f4-245d-4723-9c15-1e6154777cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.067726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079c1f4-245d-4723-9c15-1e6154777cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.077088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-494681b3-dc05-4a03-919b-f71450355267 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.078317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-494681b3-dc05-4a03-919b-f71450355267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.086703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 779/1555] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.087270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494681b3-dc05-4a03-919b-f71450355267 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.087479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8079c1f4-245d-4723-9c15-1e6154777cd6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.088564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8079c1f4-245d-4723-9c15-1e6154777cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.098831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494681b3-dc05-4a03-919b-f71450355267 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.100078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 777/1556] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.101525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.101816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.102093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.103867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c603b1-b583-4888-97e0-6e41adcf1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.104637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c603b1-b583-4888-97e0-6e41adcf1b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.104934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c603b1-b583-4888-97e0-6e41adcf1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.105274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c603b1-b583-4888-97e0-6e41adcf1b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.106008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c603b1-b583-4888-97e0-6e41adcf1b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.108279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c603b1-b583-4888-97e0-6e41adcf1b0f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.119939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 780/1557] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:53.121378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522be65-689c-430a-8d7b-23dd39cc594c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.121669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522be65-689c-430a-8d7b-23dd39cc594c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.121944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522be65-689c-430a-8d7b-23dd39cc594c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.124767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.126090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.134182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 778/1558] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.134994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.145887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5522be65-689c-430a-8d7b-23dd39cc594c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.147173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5522be65-689c-430a-8d7b-23dd39cc594c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.151790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc43ce-d7a8-413b-a349-e527d44d8de0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.159774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 781/1559] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.159994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.160125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec578-a738-4047-8499-ad971bc37534 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.160125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec578-a738-4047-8499-ad971bc37534 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.160125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec578-a738-4047-8499-ad971bc37534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.160386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.160386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.160386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.160386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.161196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.172268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 779/1560] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:53.173145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.177962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9dfec578-a738-4047-8499-ad971bc37534 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.179128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9dfec578-a738-4047-8499-ad971bc37534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.184482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57fdb9-68be-403f-9521-6e1f94bc9819 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.187924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 782/1561] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.189259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.189546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.189820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.193668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.194277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.194556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.194828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.195474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:53.195776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.197416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967f08d-3d44-4e38-aff8-b5d8cba6586b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.210535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.211507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 780/1562] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:53.211767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.213073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.213362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.213712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.221059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 783/1563] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.221951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.232452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d65fb-3a1c-4b71-9e91-06a24b8fe1cd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.234354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.235584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.237604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.238255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.238663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.238977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.240026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.241819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.245290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 781/1564] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:53.246377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.258117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 784/1565] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:53.258117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.262695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff225af-9389-445e-80f1-8ef06c2e13d5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.267374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43b43e-5db6-41a7-8ed4-2cd52dd40557 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.268376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.268984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.269264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.269536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.270180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:53.270483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.272119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.272255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.272748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.273025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.273293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.273933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:53.274228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.275806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.284230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 782/1566] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:53.285079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.293381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 785/1567] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:53.294501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.303518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1720d7-2bd2-47ac-9c6c-6507d8d56b19 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.305334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d09fa-8551-4556-8217-0438dfa9ff41 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.308575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.309194 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.309471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.309799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.310354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.310467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.311713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.311993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.312301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.312962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:53.313055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.313269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.314894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.322408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:53.328400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 783/1568] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:53.336813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 786/1569] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:53.336813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.337104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.341977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48580cc4-3336-4bc7-8ec2-20de099892a4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.342980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bd2a8-43b0-4c0c-ab3d-434993241249 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.347159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.347718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.347875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.348023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.348320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.348431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.348633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.348901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.349122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:53.349332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.349528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.350846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.351410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.361416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:53.364568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 787/1570] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:53.365514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.377894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a5018-be9d-459e-a352-25c142d3f3ac admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.387416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.388037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.388372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.388670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.390316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 784/1571] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.390504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:15:53.390504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.391117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.392126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.413279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 788/1572] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:15:53.414287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.434614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938743a-1023-478b-91be-c9588deb7ed5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.480947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3628b69-e3ca-4f46-84bc-639189c99060 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.486464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.487212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.487489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.487911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] PATH_INFO: `/v3/projects/129f3d9f33ad4b209039244eb09599e9/users/d045f032dbfb450aa9c087ada1637ee0/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.497364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.498072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.498284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.498553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.499195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.501173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.503398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129f3d9f33ad4b209039244eb09599e9, user_id=d045f032dbfb450aa9c087ada1637ee0, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.504474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:15:53.504474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:15:53.504474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:15:53.506412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.521794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 785/1573] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:53 2022] PUT /identity/v3/projects/129f3d9f33ad4b209039244eb09599e9/users/d045f032dbfb450aa9c087ada1637ee0/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:53.522875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.537616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 789/1574] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.538465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.595608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33051fd2-49f6-467a-b28c-4d2af34f1fd4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.613582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d8cb2-5448-48c5-bddd-838c1b3b6251 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.618375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.619037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.619339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.619609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] PATH_INFO: `/v3/projects/aab35f7119e44d3c8da92076c7e823db/users/976558170ae64a9cbbd6930e7c3b048a/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.635905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab35f7119e44d3c8da92076c7e823db, user_id=976558170ae64a9cbbd6930e7c3b048a, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.639211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.656968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 790/1575] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:53 2022] PUT /identity/v3/projects/aab35f7119e44d3c8da92076c7e823db/users/976558170ae64a9cbbd6930e7c3b048a/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:53.657783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.659988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.661466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.662090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.662407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.663501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.667013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.702390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 786/1576] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/projects => generated 389 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.703219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.705501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:53.711181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:53.735346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa442cb-74f8-4f50-b9fa-60650fdb3c63 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.775876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dc459-7118-4c4b-ae3f-c4f56940b3a0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.780919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.781532 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.781810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.782078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.782699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.784653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.802308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.802939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.803212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.803486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.804109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.806492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.818415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 787/1577] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.818415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.838636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 791/1578] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.839460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:53.882619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbbe2d-50fd-45b1-bdc1-6b2344eb597f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.913255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39833c-b694-4500-b87f-0d0c9ef43eef admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:53.921163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68567a85-8a96-4cbd-b7e2-6054a7d923a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.922132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68567a85-8a96-4cbd-b7e2-6054a7d923a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.922602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68567a85-8a96-4cbd-b7e2-6054a7d923a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.923159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68567a85-8a96-4cbd-b7e2-6054a7d923a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.924159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68567a85-8a96-4cbd-b7e2-6054a7d923a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.927447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68567a85-8a96-4cbd-b7e2-6054a7d923a0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.948080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:53.949234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.949979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.950442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.951688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:53.955957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:53.962010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 792/1579] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.963556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9731-e066-4112-ae24-d6a2ac57a62e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:53.963841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9731-e066-4112-ae24-d6a2ac57a62e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:53.964116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9731-e066-4112-ae24-d6a2ac57a62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:53.985297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2b7a9731-e066-4112-ae24-d6a2ac57a62e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:53.986526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b7a9731-e066-4112-ae24-d6a2ac57a62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:53.992717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 788/1580] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:53.992717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.002406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cd89-bff8-415d-8608-6270d2baa76d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.038858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:54.051677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:54.064485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 793/1581] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:54.065765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.066045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.066315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.079047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.079670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.080463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.080463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.081027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.083016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.086098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:54.087303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:54.096080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 794/1582] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:54.100083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.111671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b940c-fc89-42b3-9b5d-e6ce146772db None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.116043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 789/1583] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:53 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:54.116043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.119786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.120438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.120718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.120990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.125271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.125271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.125533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc3c06-8346-4231-9fa5-c86301906767 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.130348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.131024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.131362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.131639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.132535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.135752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.148448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 795/1584] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:54 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:54.150080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.167103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 790/1585] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:54 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:54.167103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.167503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763f734-8865-422f-8a00-3e37319d330a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.173129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.173752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.174025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.174289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.174911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.176888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.178366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a3e6-4801-4476-af99-ad964beb3d8a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.183028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.183631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.183910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.184183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.184827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.187540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.205555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 796/1586] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:54 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:54.206597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.216548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85254a10-4c55-4fa0-9dde-676869a4c6ce admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.218474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 791/1587] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:54 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:54.219381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.223393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.223999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.224319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.224597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.225252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.226900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.238343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 797/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.239212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.298960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306b166-b629-4d32-ba09-bb18736d1cef admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.299949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b030a6e-55a1-4930-b365-d1a075f39347 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.304746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.305302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.305535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.305708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.305882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.306029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.306215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.306604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.316673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.316967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.318118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.318312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.330487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 798/1589] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:54.331621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.332372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 792/1590] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:54.333458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.343241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a5df-4289-47c9-b642-bd64af0696f2 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.345318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4fae-3fe2-4808-a02c-67fa8e66f37e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.349239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.349899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.350180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.350454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.351257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.352933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.353864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.354934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.355505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.356067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.358606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.361435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.365251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 799/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.366122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.381777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 793/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.384757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b86e0-dfec-4f70-b52c-ec7382a809c3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.384872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.387785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.388500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.389896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.390187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.390841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.392693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.398877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24a0cd-003c-4de9-964f-44cb56edc24c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.407899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 800/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.408703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.408843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.409845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.410479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.411055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.412240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.415156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.420801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c837e-0cab-48b3-9798-e523681557c6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.427144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.427755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.428017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.428355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.428983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.430925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.431151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 794/1594] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.431924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.443127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718efc6-71bd-44c6-a414-c72c3e9903d8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.448017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.448679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.448954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.449225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.454149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.454378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 801/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.454378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.454567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.464584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 795/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.465123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462a651-36c1-4064-b624-090770113ab7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.469827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.472498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.473463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.473909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.474311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.475113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.477231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.484660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f09eb-cb13-4173-8fff-c1a110147c38 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.490214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 802/1597] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.491456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.494321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.495632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.495812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.496716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.497886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.500899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.505349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3e19a-3758-424e-be38-dd0beca8b6ec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.511866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.512911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 796/1598] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.514213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.514426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.515266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.515964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.517255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.520085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.531615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b31d9-ac6d-4aca-8a76-7d8a63deebf3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.537370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.538009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.538284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.538549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.543647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.543647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.543953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 803/1599] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.543953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.557075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 797/1600] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.557607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.562997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa720b-ad28-4e64-85a7-e650cdfa4d63 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.568315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.569388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.569525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698b249-d625-4dec-99b7-9468e2cdc5c4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.569628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.569628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.570448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.572245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.575573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.577120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.577469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.577816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] PATH_INFO: `/v3/projects/f1e1b718c6b944cfb3f5122dc4d25501/users/0e6c8cee157e45869cc079bb211513b5/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.586882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 804/1601] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:54.588256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.600279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e1b718c6b944cfb3f5122dc4d25501, user_id=0e6c8cee157e45869cc079bb211513b5, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.601150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0340ae0-c40e-449c-b2b1-7321bd2a141a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.603753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.606384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.607172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.607981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.608244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] PATH_INFO: `/v3/projects/9bde4e46648043d1bcf041be06ae18cb/users/c5ddb9b133384f25bc0d67cbc1c39b8b/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.619489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 798/1602] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:54 2022] PUT /identity/v3/projects/f1e1b718c6b944cfb3f5122dc4d25501/users/0e6c8cee157e45869cc079bb211513b5/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:54.621294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.633741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bde4e46648043d1bcf041be06ae18cb, user_id=c5ddb9b133384f25bc0d67cbc1c39b8b, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.637017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.643628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:54.658222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 805/1603] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:54 2022] PUT /identity/v3/projects/9bde4e46648043d1bcf041be06ae18cb/users/c5ddb9b133384f25bc0d67cbc1c39b8b/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:54.663212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.690732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:54.744774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4962cb-737a-46d4-b32d-56dfcc29785b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.746623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682626cc-a40a-4d80-aea5-918cee4a50f0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.752104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.752451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.752876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.753057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.753185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.753363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.753480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] PATH_INFO: `/v3/projects/be810e9efc4f4214b473f76da7e2714b/users/dae3738120794f9fa52c8054e3c462ed/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.755850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] PATH_INFO: `/v3/projects/b3cf862a38da43c081db5e171efe9459/users/faf58e78faae461fa0e7b75eb30c6c61/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.771752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be810e9efc4f4214b473f76da7e2714b, user_id=dae3738120794f9fa52c8054e3c462ed, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.774713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.778271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cf862a38da43c081db5e171efe9459, user_id=faf58e78faae461fa0e7b75eb30c6c61, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.781969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:54.791514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 799/1604] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:54 2022] PUT /identity/v3/projects/be810e9efc4f4214b473f76da7e2714b/users/dae3738120794f9fa52c8054e3c462ed/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:54.796067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.808751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 806/1605] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:54 2022] PUT /identity/v3/projects/b3cf862a38da43c081db5e171efe9459/users/faf58e78faae461fa0e7b75eb30c6c61/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:54.809992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:54.880367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dedba-8f41-4bb9-83e5-0c92834e1d95 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.921666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65c5f-ec0e-402f-a4e4-ab90aa78579c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:54.933792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac0e0e-c577-4366-88b1-8d5a7d97ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:54.934547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac0e0e-c577-4366-88b1-8d5a7d97ff5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:54.934984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac0e0e-c577-4366-88b1-8d5a7d97ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:54.935559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac0e0e-c577-4366-88b1-8d5a7d97ff5d admin admin] PATH_INFO: `/v3/projects/bcd0b12fb80a401690dd728c318968a0/users/8684a435be704fb79600b32813b276b3/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:54.969385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac0e0e-c577-4366-88b1-8d5a7d97ff5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd0b12fb80a401690dd728c318968a0, user_id=8684a435be704fb79600b32813b276b3, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:54.975695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac0e0e-c577-4366-88b1-8d5a7d97ff5d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.005955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88645db-4799-47e9-8342-8d6c536107fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:55.007944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88645db-4799-47e9-8342-8d6c536107fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.008107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 807/1606] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:54 2022] PUT /identity/v3/projects/bcd0b12fb80a401690dd728c318968a0/users/8684a435be704fb79600b32813b276b3/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:55.009542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d3de7-9062-4894-9143-e7fed2c06f42 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.011155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d3de7-9062-4894-9143-e7fed2c06f42 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.011576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d3de7-9062-4894-9143-e7fed2c06f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.019955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88645db-4799-47e9-8342-8d6c536107fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.019955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88645db-4799-47e9-8342-8d6c536107fb admin admin] PATH_INFO: `/v3/projects/ae2c606234664b83b82eaf8f7c577899/users/5e75d233c3b14229a73f96709e5cb2f8/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.060139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88645db-4799-47e9-8342-8d6c536107fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2c606234664b83b82eaf8f7c577899, user_id=5e75d233c3b14229a73f96709e5cb2f8, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:55.060429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d84d3de7-9062-4894-9143-e7fed2c06f42 None None] MFA Rules not processed for user `0e6c8cee157e45869cc079bb211513b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.061779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d84d3de7-9062-4894-9143-e7fed2c06f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.063454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88645db-4799-47e9-8342-8d6c536107fb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.082459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 800/1607] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:54 2022] PUT /identity/v3/projects/ae2c606234664b83b82eaf8f7c577899/users/5e75d233c3b14229a73f96709e5cb2f8/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:55.096851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3933180-5ada-44f2-baa1-6433d65f8be4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.097199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3933180-5ada-44f2-baa1-6433d65f8be4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.097727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3933180-5ada-44f2-baa1-6433d65f8be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.133696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c3933180-5ada-44f2-baa1-6433d65f8be4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.135177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c3933180-5ada-44f2-baa1-6433d65f8be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.166238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:55.174361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 808/1608] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:55.175682 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0074e-b615-4539-b0c8-c23d8babd021 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.176017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0074e-b615-4539-b0c8-c23d8babd021 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.176337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0074e-b615-4539-b0c8-c23d8babd021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.211924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cdf0074e-b615-4539-b0c8-c23d8babd021 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.213720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cdf0074e-b615-4539-b0c8-c23d8babd021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.224825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:55.277238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 801/1609] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:55.280147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e655-beb8-4480-82f4-42ed0c24ad33 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.280725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e655-beb8-4480-82f4-42ed0c24ad33 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.281274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e655-beb8-4480-82f4-42ed0c24ad33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.297904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 809/1610] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:55.302717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.303075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.303420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.316441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1770e655-beb8-4480-82f4-42ed0c24ad33 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.318483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1770e655-beb8-4480-82f4-42ed0c24ad33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.335056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 802/1611] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:55.335333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.335933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.336491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.341036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] MFA Rules not processed for user `8684a435be704fb79600b32813b276b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.343561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.391697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] MFA Rules not processed for user `5e75d233c3b14229a73f96709e5cb2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.393721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.455065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 810/1612] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:55.455766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:55.502714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 803/1613] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:55.503510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:55.513395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:55.535984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4f1a1-c3cb-423d-a44f-d7fa637afa11 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:55.544056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:55.545340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.545340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.545678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.613202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71a960-ff80-48cb-9cc0-930b42e0f38b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:55.617533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:55.618164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.618444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.618733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.630844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:55.632461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.647945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 804/1614] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:55.649070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:55.658924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23980bef-a2b7-4310-b553-718028a7a627 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:55.663333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:55.663979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.664472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.664760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.665580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:55.667728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.671833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:55.686585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:55.688055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.696750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 811/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:55.698153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:55.714552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 805/1616] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:55.715367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:55.725101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb138a-8594-4e7c-90d0-c5f05576284e service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:55.732923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9baf9-3cb0-4c58-a930-73d369acf8e7 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:55.738910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:55.741395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.742003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.742507 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.744985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cee70-f49c-4039-97ba-de0dc441d59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:55.745959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cee70-f49c-4039-97ba-de0dc441d59e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.747278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cee70-f49c-4039-97ba-de0dc441d59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.748895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cee70-f49c-4039-97ba-de0dc441d59e admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.750172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cee70-f49c-4039-97ba-de0dc441d59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:55.753505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cee70-f49c-4039-97ba-de0dc441d59e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.770969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:55.772891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:55.808692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 806/1617] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:55.808956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 812/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:55.811225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a11e1-7d7e-4f7a-b521-7283802da8d5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.811542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a11e1-7d7e-4f7a-b521-7283802da8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.811828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a11e1-7d7e-4f7a-b521-7283802da8d5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.813555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:55.819769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 807/1619] 10.222.0.16 () {54 vars in 962 bytes} [Thu Aug 18 11:15:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:15:55.824679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:55.824996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:55.825230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:55.835840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8469dd-c669-40a5-975f-950ef7f5eb00 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:55.880544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:55.930611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:55.934467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:56.031170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:56.033454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.033917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.034390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.038815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:56.047509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:56.119310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 813/1620] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/users => generated 294 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:15:56.126886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:56.150572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a190b-3ad0-4e5f-9a2f-d07c0e246d15 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:56.167905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:56.169109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.169109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.169452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.170922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:56.173767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:56.192259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 814/1621] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:15:56.203593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 808/1622] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Aug 18 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:56.206420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:56.206906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:56.241116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5964-f220-40fb-8d68-222aaf970012 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:56.242503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fcfaf-9c50-4e51-b257-aa2466a1586b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:56.247846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:56.249327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.249327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.249854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] PATH_INFO: `/v3/projects/19838c946271462a9d25b3356beb9575/users/1803033809f4487bbb488636bca651c2/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.259356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:56.260764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.261114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.261931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.274103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19838c946271462a9d25b3356beb9575, user_id=1803033809f4487bbb488636bca651c2, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:56.278727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:56.314645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 815/1623] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:56 2022] PUT /identity/v3/projects/19838c946271462a9d25b3356beb9575/users/1803033809f4487bbb488636bca651c2/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:56.316617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:56.413556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:56.418764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:56.446265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 809/1624] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:56.469654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629935e8-3c2f-41ea-ba4e-753a3dcdc728 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:56.484608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f98fb8-e344-4f88-96a3-2c3f5561b5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:56.486674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98fb8-e344-4f88-96a3-2c3f5561b5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.489808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98fb8-e344-4f88-96a3-2c3f5561b5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.490955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98fb8-e344-4f88-96a3-2c3f5561b5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.491704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f98fb8-e344-4f88-96a3-2c3f5561b5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:56.496292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f98fb8-e344-4f88-96a3-2c3f5561b5a5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:56.525572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 816/1625] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:56.530693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:56.552870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfb110-1eee-4909-8f25-5e83a1fc9498 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:56.566898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:56.568410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.570105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.571692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] PATH_INFO: `/v3/projects/19838c946271462a9d25b3356beb9575/users/1803033809f4487bbb488636bca651c2/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.577724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19838c946271462a9d25b3356beb9575, user_id=1803033809f4487bbb488636bca651c2, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:56.587901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:56.603926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 810/1626] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:56 2022] PUT /identity/v3/projects/19838c946271462a9d25b3356beb9575/users/1803033809f4487bbb488636bca651c2/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:56.615279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a03e9-107d-4a42-8fbc-b6bde474ea5f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:56.615961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a03e9-107d-4a42-8fbc-b6bde474ea5f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:56.616423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a03e9-107d-4a42-8fbc-b6bde474ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:56.681594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6c0a03e9-107d-4a42-8fbc-b6bde474ea5f None None] MFA Rules not processed for user `1803033809f4487bbb488636bca651c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:56.683319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6c0a03e9-107d-4a42-8fbc-b6bde474ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:56.761367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:56.908541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 817/1627] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:15:56.922189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:57.059902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dbef8-a3bb-461e-9d00-2a75c98bde9a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:57.071139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c2130-8dfe-4fe9-b0d3-502e9c520c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:57.071771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c2130-8dfe-4fe9-b0d3-502e9c520c2c service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.072092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c2130-8dfe-4fe9-b0d3-502e9c520c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.072388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c2130-8dfe-4fe9-b0d3-502e9c520c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.115521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:57.173305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c2130-8dfe-4fe9-b0d3-502e9c520c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:57.176387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c2130-8dfe-4fe9-b0d3-502e9c520c2c service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:57.191240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 811/1628] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:57.441013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0103-d1cf-4702-8842-c27dcf4e27ca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.441418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0103-d1cf-4702-8842-c27dcf4e27ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.441939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0103-d1cf-4702-8842-c27dcf4e27ca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.444281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 818/1629] 10.222.0.16 () {54 vars in 962 bytes} [Thu Aug 18 11:15:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:15:57.455471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe8fa-8200-4dde-a72a-d748f045fd81 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.455968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe8fa-8200-4dde-a72a-d748f045fd81 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.456114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe8fa-8200-4dde-a72a-d748f045fd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.500333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-15dbe8fa-8200-4dde-a72a-d748f045fd81 None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:57.502357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-15dbe8fa-8200-4dde-a72a-d748f045fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:57.733232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 812/1630] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Aug 18 11:15:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:57.740131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3571ccf-016e-49aa-be75-9d969d5f24f0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.740765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3571ccf-016e-49aa-be75-9d969d5f24f0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.740765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3571ccf-016e-49aa-be75-9d969d5f24f0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.742884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 819/1631] 10.222.0.16 () {54 vars in 912 bytes} [Thu Aug 18 11:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:15:57.751344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04e6c-db7d-4815-a5d1-14d98ef693ca None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.751606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04e6c-db7d-4815-a5d1-14d98ef693ca None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.751905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04e6c-db7d-4815-a5d1-14d98ef693ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.781678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dfe04e6c-db7d-4815-a5d1-14d98ef693ca None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:57.783238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dfe04e6c-db7d-4815-a5d1-14d98ef693ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:57.836765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 813/1632] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:15:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:57.851512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3571ccf-016e-49aa-be75-9d969d5f24f0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:57.872282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3571ccf-016e-49aa-be75-9d969d5f24f0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:57.879969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbb25a-9c83-4f96-a5b1-378a8c2ca45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:57.880625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbb25a-9c83-4f96-a5b1-378a8c2ca45b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.880916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbb25a-9c83-4f96-a5b1-378a8c2ca45b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.881190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbb25a-9c83-4f96-a5b1-378a8c2ca45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.904857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbb25a-9c83-4f96-a5b1-378a8c2ca45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:57.906595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbb25a-9c83-4f96-a5b1-378a8c2ca45b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:57.940607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:57.940892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:57.941178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:57.943098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 820/1633] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:58.035604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] MFA Rules not processed for user `c5ddb9b133384f25bc0d67cbc1c39b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:58.037408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:58.088654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e30ce-8690-4a52-8d07-607791ef5c13 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.088857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e30ce-8690-4a52-8d07-607791ef5c13 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.089094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e30ce-8690-4a52-8d07-607791ef5c13 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.092943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 821/1634] 10.222.0.16 () {54 vars in 912 bytes} [Thu Aug 18 11:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:15:58.101610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.102191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.102631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.127382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:58.129031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:58.160675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 822/1635] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:15:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:15:58.175903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:58.188108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d592dc-b136-4728-beb3-1bb22b249523 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:58.200246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:58.204532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.204894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.207272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.244723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 814/1636] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:57 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:15:58.246485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:58.248949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:58.252089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:58.282802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 823/1637] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:15:58.363179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:58.403203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456eccf-a934-423e-9b51-58170d8a770e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:58.415109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:58.415109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.415109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.415109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.416654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:58.417845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:58.421114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:58.437541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcfb5f-c685-4c39-83fb-9eb800b006ac service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:58.457235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:58.458096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.458475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.458842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] PATH_INFO: `/v3/users/5e75d233c3b14229a73f96709e5cb2f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.461984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e75d233c3b14229a73f96709e5cb2f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:58.464240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:58.504552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 815/1638] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:15:58 2022] POST /identity/v3/projects => generated 405 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:15:58.506046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:58.611424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e75d233c3b14229a73f96709e5cb2f8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:15:58.613892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e75d233c3b14229a73f96709e5cb2f8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:15:58.615541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e75d233c3b14229a73f96709e5cb2f8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:15:58.648731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79175c2-5c9f-4276-91a4-cea288efbd4f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:58.662206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:58.664650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.665028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e75d233c3b14229a73f96709e5cb2f8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:15:58.665153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.665724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.666182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:58.672170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:58.692372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-87f5e14e-8b6d-4d38-80db-71076628bcad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e75d233c3b14229a73f96709e5cb2f8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:15:58.772711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 816/1639] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:15:58 2022] POST /identity/v3/users => generated 309 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:15:58.772711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:58.772711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:58.780386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 824/1640] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:15:58 2022] DELETE /identity/v3/users/5e75d233c3b14229a73f96709e5cb2f8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:15:58.791440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:58.792418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:58.793156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:58.809381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e266e1-02d5-4a4c-92bf-c0c0cf003865 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:58.834861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:15:58.837518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:59.004559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:59.005433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:59.005813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:59.006930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:59.007711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:59.012386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:59.040875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:59.049321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 817/1641] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:15:59.055010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:59.071972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 825/1642] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:15:59.086386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d7f6-c0a9-4d37-acb3-3cef1ff413f1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:59.105175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:59.106146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:59.106647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:59.107036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] PATH_INFO: `/v3/projects/52547696f2734798a80a82e21966fcca/users/db30814618b44fd592869ab92b985731/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:59.133257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52547696f2734798a80a82e21966fcca, user_id=db30814618b44fd592869ab92b985731, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:59.137151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:59.161256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 818/1643] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:59 2022] PUT /identity/v3/projects/52547696f2734798a80a82e21966fcca/users/db30814618b44fd592869ab92b985731/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:15:59.164963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:59.179049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab00fc-32d3-4f41-8097-45069035b060 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:59.181376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:59.184646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:15:59.259839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:15:59.332903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a10b-5607-4eb7-af18-87bc10900a7b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:59.341725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:59.342812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:59.343224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:59.343624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:59.351354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:59.351988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:59.352331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:59.352609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:59.355486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:59.357166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:59.362490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:59.364228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:59.382074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 819/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:15:59.393448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 826/1645] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:15:59.398254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:15:59.416365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c8936-9ded-4f6a-a367-1c0782ad42f9 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:15:59.426550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3754c-54f7-402a-8b47-645d46b2fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:15:59.427423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3754c-54f7-402a-8b47-645d46b2fb52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:15:59.427809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3754c-54f7-402a-8b47-645d46b2fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:15:59.428133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3754c-54f7-402a-8b47-645d46b2fb52 admin admin] PATH_INFO: `/v3/projects/52547696f2734798a80a82e21966fcca/users/db30814618b44fd592869ab92b985731/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:15:59.430456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3754c-54f7-402a-8b47-645d46b2fb52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52547696f2734798a80a82e21966fcca, user_id=db30814618b44fd592869ab92b985731, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:15:59.440721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3754c-54f7-402a-8b47-645d46b2fb52 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:15:59.467979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 820/1646] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:15:59 2022] PUT /identity/v3/projects/52547696f2734798a80a82e21966fcca/users/db30814618b44fd592869ab92b985731/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:01.471406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:01.474222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:01.496838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe31f5-0b01-4ff4-8900-29ffd5e37bba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:01.656436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:01.657305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:01.657597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:01.657872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] PATH_INFO: `/v3/projects/ae2c606234664b83b82eaf8f7c577899` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:01.661409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2c606234664b83b82eaf8f7c577899)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:01.663515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:16:01.663515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 18 11:16:01.663515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: Aug 18 11:16:01.664794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:01.820694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2c606234664b83b82eaf8f7c577899', 'request_id': 'req-86657dc6-1be9-4ae4-b5ee-30002a169ec5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:01.833499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2c606234664b83b82eaf8f7c577899', 'request_id': 'req-86657dc6-1be9-4ae4-b5ee-30002a169ec5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:01.855316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2c606234664b83b82eaf8f7c577899', 'request_id': 'req-86657dc6-1be9-4ae4-b5ee-30002a169ec5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:01.874454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] The token cache is being invalidate because project ae2c606234664b83b82eaf8f7c577899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:01.875402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:01.876320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86657dc6-1be9-4ae4-b5ee-30002a169ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:01.878975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 827/1647] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:01 2022] DELETE /identity/v3/projects/ae2c606234664b83b82eaf8f7c577899 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:01.884057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:01.884649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:01.884649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:01.932790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:01.934715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:02.080302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 821/1648] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:02.087938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.088318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.088535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.132373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:02.137635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:02.162902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 828/1649] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:02.168279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:02.190208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cec80e-55f4-4372-bde2-86bb147f8d71 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:02.205367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:02.206197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.206721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.207118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.208229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:02.208828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:02.215959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6136f-5ba8-4347-b957-531d8320f74f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:02.250353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 822/1650] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:02.254489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:02.266318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819e745-b181-4fe1-a485-51a790d62ed3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:02.290808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:02.291411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.291793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.292162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.292946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:02.295320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:02.312762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.313278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.313651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.379584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:02.389472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] MFA Rules not processed for user `db30814618b44fd592869ab92b985731`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:02.391989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:02.393037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 829/1651] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:02 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:02.396752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:02.578071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6196953-f715-42b1-9082-a048bc8ee3d7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:02.597864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 823/1652] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:02.606641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:02.608080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.608766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.609237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.610054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:02.612801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:02.641299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 830/1653] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:02 2022] POST /identity/v3/users => generated 303 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:02.644788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:02.710529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d2a7d-9ea2-4adb-97a7-e92b0b9addb6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:02.718264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:02.721303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.725298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.725298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.731274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:02.733653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:02.749845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 824/1654] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:02.751175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:02.765092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a83ed-8f5f-426c-9461-9f9bff883d5a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:02.778926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7dc89a-f540-4c80-88ec-f50faea97422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:02.780226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dc89a-f540-4c80-88ec-f50faea97422 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.788597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dc89a-f540-4c80-88ec-f50faea97422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.788940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dc89a-f540-4c80-88ec-f50faea97422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.789707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7dc89a-f540-4c80-88ec-f50faea97422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:02.791477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7dc89a-f540-4c80-88ec-f50faea97422 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:02.830736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 831/1655] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:02.832158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:02.845837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80123f8-00db-47ba-ad44-28cd5ff1060e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:02.853200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:02.854313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.855107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.855607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] PATH_INFO: `/v3/projects/91e84d72229e441ebf93cf99b28999ce/users/860be0e57734414cac66bf693a4c896a/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.877646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e84d72229e441ebf93cf99b28999ce, user_id=860be0e57734414cac66bf693a4c896a, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:02.881018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:02.886204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:02.914497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 825/1656] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:02 2022] PUT /identity/v3/projects/91e84d72229e441ebf93cf99b28999ce/users/860be0e57734414cac66bf693a4c896a/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:02.915765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:02.916053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:02.916356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:02.954537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] MFA Rules not processed for user `860be0e57734414cac66bf693a4c896a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:02.957045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:03.100880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 832/1657] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:03.103188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:03.214038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a2aa2-03a2-4611-a60a-d44d27b953fc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:03.221475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:03.222272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.222669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.223207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.224016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:03.227413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:03.293146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 826/1658] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:03 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:03.295194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:03.311539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a2db2-d6d8-4fe5-b9e8-5c3e63c951f3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:03.427403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:03.430145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.430500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.430956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.431743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:03.435277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:03.456254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:03.480091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:03.490343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 833/1659] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:03 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:03.496380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599418-ec3f-485a-af95-6a78d7a6b419 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:03.504777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:03.505721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.506104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.506480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.507300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:03.509632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:03.527407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 827/1660] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:03.532273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:03.557303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8025ce-1749-4b71-8546-9c2358851a01 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:03.567764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:03.569008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.569581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.570074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] PATH_INFO: `/v3/projects/86563b6fb9e94d2096e628a9c1f05cd8/users/559ed67a99e24c39955c8ed3ced19fa5/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.598902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86563b6fb9e94d2096e628a9c1f05cd8, user_id=559ed67a99e24c39955c8ed3ced19fa5, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:03.609091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:03.644454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:03.644539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 834/1661] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:03 2022] PUT /identity/v3/projects/86563b6fb9e94d2096e628a9c1f05cd8/users/559ed67a99e24c39955c8ed3ced19fa5/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:03.663581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1a479-8f19-4aa1-8a92-1f2b0f8ab7df admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:03.773879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9427d-d46a-468a-bac4-7373f292d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:03.775806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9427d-d46a-468a-bac4-7373f292d2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.776802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9427d-d46a-468a-bac4-7373f292d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.777453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9427d-d46a-468a-bac4-7373f292d2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.778728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9427d-d46a-468a-bac4-7373f292d2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:03.782772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9427d-d46a-468a-bac4-7373f292d2f1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:03.820954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 828/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:03.822434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:03.847998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3790d3-6ce7-4d5d-8da1-33496f092d4d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:03.865189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:03.866300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.866523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.868351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] PATH_INFO: `/v3/projects/86563b6fb9e94d2096e628a9c1f05cd8/users/559ed67a99e24c39955c8ed3ced19fa5/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.874193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] RBAC: Authorizing `identity:create_grant(project_id=86563b6fb9e94d2096e628a9c1f05cd8, user_id=559ed67a99e24c39955c8ed3ced19fa5, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:03.878563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:03.891476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 835/1663] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:03 2022] PUT /identity/v3/projects/86563b6fb9e94d2096e628a9c1f05cd8/users/559ed67a99e24c39955c8ed3ced19fa5/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:03.909816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ebf8b-ef76-4929-83b9-4113cced191e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:03.910352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ebf8b-ef76-4929-83b9-4113cced191e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:03.910352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ebf8b-ef76-4929-83b9-4113cced191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:03.954288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-506ebf8b-ef76-4929-83b9-4113cced191e None None] MFA Rules not processed for user `559ed67a99e24c39955c8ed3ced19fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:03.955666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-506ebf8b-ef76-4929-83b9-4113cced191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:04.103370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:04.128209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 829/1664] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:04.137148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:04.140344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:04.282494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d025851-f94a-47dc-9efa-0e4f22a8effb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:04.290601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a9f4a-af98-4f4a-8bac-0d08ae6cf679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:04.291354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a9f4a-af98-4f4a-8bac-0d08ae6cf679 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:04.291736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a9f4a-af98-4f4a-8bac-0d08ae6cf679 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:04.292131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a9f4a-af98-4f4a-8bac-0d08ae6cf679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:04.304499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a9f4a-af98-4f4a-8bac-0d08ae6cf679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:04.307473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a9f4a-af98-4f4a-8bac-0d08ae6cf679 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:04.320395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 836/1665] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:04 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:04.529112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77318be0-8459-42c7-9fb8-38f4fcb51bc7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:04.529112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77318be0-8459-42c7-9fb8-38f4fcb51bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:04.529112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77318be0-8459-42c7-9fb8-38f4fcb51bc7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:04.531243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 830/1666] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:16:04.572889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a321cb-c4de-43c0-9abd-34124314cfec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:04.573176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a321cb-c4de-43c0-9abd-34124314cfec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:04.573453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a321cb-c4de-43c0-9abd-34124314cfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:04.633563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a9a321cb-c4de-43c0-9abd-34124314cfec None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:04.640632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9a321cb-c4de-43c0-9abd-34124314cfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:04.829383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 837/1667] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:16:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:04.953528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77318be0-8459-42c7-9fb8-38f4fcb51bc7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:04.955290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-77318be0-8459-42c7-9fb8-38f4fcb51bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:04.985982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77318be0-8459-42c7-9fb8-38f4fcb51bc7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:05.010257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:05.011157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:05.011586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:05.011993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:05.034779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:05.037433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:05.085413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 831/1668] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:05.752656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5b4a-758e-41e5-b17d-a744fc286377 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:05.756526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5b4a-758e-41e5-b17d-a744fc286377 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:05.756526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5b4a-758e-41e5-b17d-a744fc286377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:05.756749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:05.841012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9bfc5b4a-758e-41e5-b17d-a744fc286377 None None] MFA Rules not processed for user `976558170ae64a9cbbd6930e7c3b048a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:05.842855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9bfc5b4a-758e-41e5-b17d-a744fc286377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:05.961989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a052f0-af79-471e-88f4-7ac512999a3f service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:05.969586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:05.970526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:05.971006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:05.971470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] PATH_INFO: `/v3/users/860be0e57734414cac66bf693a4c896a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:05.973016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860be0e57734414cac66bf693a4c896a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:05.974398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 11:16:05.974398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Aug 18 11:16:05.974398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: Aug 18 11:16:05.976441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:06.109491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860be0e57734414cac66bf693a4c896a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:06.130715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:06.159014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860be0e57734414cac66bf693a4c896a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:06.184030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 838/1669] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:06.194487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860be0e57734414cac66bf693a4c896a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:06.194966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:06.195349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:06.195728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:06.196000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860be0e57734414cac66bf693a4c896a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:06.196824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-adcdf91a-7d8b-4272-9140-ede45d335bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860be0e57734414cac66bf693a4c896a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:06.237755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 832/1670] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:05 2022] DELETE /identity/v3/users/860be0e57734414cac66bf693a4c896a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:06.242347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:06.242722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:06.243159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:06.244007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] MFA Rules not processed for user `976558170ae64a9cbbd6930e7c3b048a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:06.247961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:06.297398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:06.298661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:06.430381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 839/1671] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:06.445069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:06.446712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:06.475632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:06.502658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 833/1672] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:06.531523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:06.533033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:06.628851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380faba1-ac21-4cab-a06a-df1d175df57d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:06.636153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:06.645708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:06.647017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:06.647456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:06.649889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:06.672592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:06.674685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:06.711172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 840/1673] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:06 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:06.714356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf713d-e41b-4a75-9b8d-f03056354502 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:06.729472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620ecdc-36e3-4a5b-9cd2-166e587b60ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:06.730288 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ecdc-36e3-4a5b-9cd2-166e587b60ad service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:06.732524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ecdc-36e3-4a5b-9cd2-166e587b60ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:06.732855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ecdc-36e3-4a5b-9cd2-166e587b60ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:06.767882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620ecdc-36e3-4a5b-9cd2-166e587b60ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:06.772414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620ecdc-36e3-4a5b-9cd2-166e587b60ad service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:06.814984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 834/1674] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:07.140452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:07.143261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:07.288268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750734-8e01-4546-9835-4ff31831d9e8 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:07.301263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:07.312848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:07.313572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:07.314051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:07.316872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:07.365754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:07.443325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:07.445108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774166c9-aba8-4c2f-8caa-209105da24ac service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:07.469671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 841/1675] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:07.515762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e42c59-848c-48c2-a06e-bf7d11be661e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:07.516395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e42c59-848c-48c2-a06e-bf7d11be661e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:07.516879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e42c59-848c-48c2-a06e-bf7d11be661e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:07.518611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 835/1676] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:16:07.557227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739b5a5-2272-4915-870d-19e3a7023a2c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:07.557605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739b5a5-2272-4915-870d-19e3a7023a2c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:07.557950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739b5a5-2272-4915-870d-19e3a7023a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:07.601758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f739b5a5-2272-4915-870d-19e3a7023a2c None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:07.605075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f739b5a5-2272-4915-870d-19e3a7023a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:07.697789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e42c59-848c-48c2-a06e-bf7d11be661e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:07.701842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-88e42c59-848c-48c2-a06e-bf7d11be661e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:07.746801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e42c59-848c-48c2-a06e-bf7d11be661e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:07.758115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:07.758831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:07.759514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:07.759800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] PATH_INFO: `/v3/users/c5ddb9b133384f25bc0d67cbc1c39b8b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:07.763544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ddb9b133384f25bc0d67cbc1c39b8b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:07.766576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:07.775172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 842/1677] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:16:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:07.824406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fae2c-8551-41ec-8229-e8df97e1e6d3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:07.826488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fae2c-8551-41ec-8229-e8df97e1e6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:07.827098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fae2c-8551-41ec-8229-e8df97e1e6d3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:07.829721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 843/1678] 10.222.0.16 () {54 vars in 961 bytes} [Thu Aug 18 11:16:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:16:07.839805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:07.841971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:07.842302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:07.887526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ddb9b133384f25bc0d67cbc1c39b8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:07.917298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ddb9b133384f25bc0d67cbc1c39b8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:07.918458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] MFA Rules not processed for user `f669319338ab4c6ea22dead596391789`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:07.929435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:07.950919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ddb9b133384f25bc0d67cbc1c39b8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:07.952336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ddb9b133384f25bc0d67cbc1c39b8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:07.953189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ddb9b133384f25bc0d67cbc1c39b8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:07.964656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 844/1679] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Aug 18 11:16:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:07.964656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:07.974233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:07.980454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 836/1680] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:07 2022] DELETE /identity/v3/users/c5ddb9b133384f25bc0d67cbc1c39b8b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:07.982016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:08.022474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e5811-558f-4f15-804c-8bdcd0189a54 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:08.125693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b54a3-af7a-4d38-bc47-76f9e41eb0c9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:08.141885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:08.142697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:08.144226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:08.146841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:08.152049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:08.178466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:08.179828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:08.180360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:08.180939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] PATH_INFO: `/v3/projects/91e84d72229e441ebf93cf99b28999ce` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:08.181380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:08.184237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e84d72229e441ebf93cf99b28999ce)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:08.185309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:08.189189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:08.215567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 837/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:08.378459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e84d72229e441ebf93cf99b28999ce', 'request_id': 'req-861877ec-8345-4649-8801-2bb7435950ca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.410532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e84d72229e441ebf93cf99b28999ce', 'request_id': 'req-861877ec-8345-4649-8801-2bb7435950ca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.414986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:08.416500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:08.416618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:08.427041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e84d72229e441ebf93cf99b28999ce', 'request_id': 'req-861877ec-8345-4649-8801-2bb7435950ca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.432559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:08.448976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] The token cache is being invalidate because project 91e84d72229e441ebf93cf99b28999ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:08.449270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.449949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.451541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6eb53-8bfb-4dd6-940b-5773957660d2 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:08.454170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 845/1682] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:07 2022] DELETE /identity/v3/projects/91e84d72229e441ebf93cf99b28999ce => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:08.458325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:08.460430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:08.568036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861877ec-8345-4649-8801-2bb7435950ca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:08.574445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:08.575329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:08.575687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:08.576949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] PATH_INFO: `/v3/users/559ed67a99e24c39955c8ed3ced19fa5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:08.578270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559ed67a99e24c39955c8ed3ced19fa5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:08.580959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:08.591017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:08.592940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:08.593447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:08.594863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] PATH_INFO: `/v3/projects/9bde4e46648043d1bcf041be06ae18cb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:08.597816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bde4e46648043d1bcf041be06ae18cb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:08.599912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:08.694744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559ed67a99e24c39955c8ed3ced19fa5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.695488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559ed67a99e24c39955c8ed3ced19fa5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.696452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559ed67a99e24c39955c8ed3ced19fa5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.708926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bde4e46648043d1bcf041be06ae18cb', 'request_id': 'req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.717874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559ed67a99e24c39955c8ed3ced19fa5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.719721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bde4e46648043d1bcf041be06ae18cb', 'request_id': 'req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.730433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59c2a1c1-bd61-4780-a7e8-475eeaea28d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559ed67a99e24c39955c8ed3ced19fa5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.751996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 846/1683] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:08 2022] DELETE /identity/v3/users/559ed67a99e24c39955c8ed3ced19fa5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:08.769376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bde4e46648043d1bcf041be06ae18cb', 'request_id': 'req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.787996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee96167-4bd9-4c76-95da-f57b590cd9e5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:08.788609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee96167-4bd9-4c76-95da-f57b590cd9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:08.789804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee96167-4bd9-4c76-95da-f57b590cd9e5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:08.791504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 847/1684] 10.222.0.16 () {54 vars in 912 bytes} [Thu Aug 18 11:16:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:16:08.804827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:08.805277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:08.805740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:08.816228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] The token cache is being invalidate because project 9bde4e46648043d1bcf041be06ae18cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:08.818276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.821887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:08.829894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 838/1685] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:08 2022] DELETE /identity/v3/projects/9bde4e46648043d1bcf041be06ae18cb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:08.831647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:08.834225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:08.899245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:08.901891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:09.035657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba83b2a-35bc-4fd9-bcaa-1a93974a08df admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:09.098877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 848/1686] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:16:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:09.119373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:09.120839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:09.128003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:09.138074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65dc87-0d93-4a7d-9851-6641372098f0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:09.148088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:09.148817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:09.152649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:09.152942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:09.166778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:09.168388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:09.194260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:09.195623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 849/1687] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:09.242084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:09.242746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:09.256514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:09.256930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] PATH_INFO: `/v3/users/db30814618b44fd592869ab92b985731` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:09.269791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db30814618b44fd592869ab92b985731)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:09.271753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:09.396946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db30814618b44fd592869ab92b985731'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:09.423645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db30814618b44fd592869ab92b985731'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:09.438731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db30814618b44fd592869ab92b985731'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:09.439638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db30814618b44fd592869ab92b985731'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:09.440460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db30814618b44fd592869ab92b985731'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:09.469222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 839/1688] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:08 2022] DELETE /identity/v3/users/db30814618b44fd592869ab92b985731 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:09.731153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:09.733275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:09.759326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4801dc-5089-4239-ad98-68803068cec3 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:09.921269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:09.926838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:09.927278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:09.929013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:09.929309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:09.929590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] PATH_INFO: `/v3/projects/86563b6fb9e94d2096e628a9c1f05cd8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:09.933445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86563b6fb9e94d2096e628a9c1f05cd8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:09.938751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:09.961366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a331492-b3c2-4c13-94d5-09a580ea548c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:09.976449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:09.977115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:09.977532 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:09.977817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] PATH_INFO: `/v3/projects/52547696f2734798a80a82e21966fcca` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:09.980500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52547696f2734798a80a82e21966fcca)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:09.995002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:10.086967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52547696f2734798a80a82e21966fcca', 'request_id': 'req-47fc185c-1d25-40ba-9f88-1e0bece58e54'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.109474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52547696f2734798a80a82e21966fcca', 'request_id': 'req-47fc185c-1d25-40ba-9f88-1e0bece58e54'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.128548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52547696f2734798a80a82e21966fcca', 'request_id': 'req-47fc185c-1d25-40ba-9f88-1e0bece58e54'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.128775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86563b6fb9e94d2096e628a9c1f05cd8', 'request_id': 'req-3688844f-2037-4242-8a41-a0b4252d6290'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.151964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86563b6fb9e94d2096e628a9c1f05cd8', 'request_id': 'req-3688844f-2037-4242-8a41-a0b4252d6290'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.161685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86563b6fb9e94d2096e628a9c1f05cd8', 'request_id': 'req-3688844f-2037-4242-8a41-a0b4252d6290'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.182587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] The token cache is being invalidate because project 52547696f2734798a80a82e21966fcca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:10.184854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.187620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47fc185c-1d25-40ba-9f88-1e0bece58e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.193398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 840/1689] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:09 2022] DELETE /identity/v3/projects/52547696f2734798a80a82e21966fcca => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:10.194958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd96f3-f3f8-4d96-b285-4ad6b490e672 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.195350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd96f3-f3f8-4d96-b285-4ad6b490e672 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.197436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] The token cache is being invalidate because project 86563b6fb9e94d2096e628a9c1f05cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:10.197906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd96f3-f3f8-4d96-b285-4ad6b490e672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.198479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.199406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3688844f-2037-4242-8a41-a0b4252d6290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:10.201172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 850/1690] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:09 2022] DELETE /identity/v3/projects/86563b6fb9e94d2096e628a9c1f05cd8 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:10.206004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f65311-716c-46aa-9f9a-1793055d6de4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.206578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f65311-716c-46aa-9f9a-1793055d6de4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.207046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f65311-716c-46aa-9f9a-1793055d6de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.236104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9ddd96f3-f3f8-4d96-b285-4ad6b490e672 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:10.237381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ddd96f3-f3f8-4d96-b285-4ad6b490e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:10.243918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2f65311-716c-46aa-9f9a-1793055d6de4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:10.247014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2f65311-716c-46aa-9f9a-1793055d6de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:10.306276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:10.340263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 841/1691] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:10.346654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.346963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.347242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.397901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:10.400312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:10.426376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 842/1692] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:10.430286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.435619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 851/1693] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:10.441603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.442229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.442229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.464458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf74bc-b48f-4315-b82f-81f63bfa18a5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:10.480826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:10.481609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.481756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.482018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.483359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:10.483678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:10.485333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:10.504385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 843/1694] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:10.504930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.518710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:10.520946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:10.523005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046001b-0cde-40a6-ac07-30cb70e2e8b9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:10.547335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 852/1695] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:10.548332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.549900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:10.550759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.551238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.551726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.552628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:10.563107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:10.565070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38b99c-2d49-4317-ae8d-10be3013bbda None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:10.584838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:10.585770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.586215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.586634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.587565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:10.588030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:10.590417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:10.625590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:10.637012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 853/1696] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:10.643523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.666237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 844/1697] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:10.667300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.670388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b3f22-959a-4de1-ac3d-4a9742932186 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:10.845449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0ea27-4d1f-44ea-bc9c-7dbd5bee2578 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:10.852952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:10.855311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.855675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.856182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.858257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:10.860009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:10.860919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:10.861386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:10.861845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:10.862660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:10.863671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:10.866489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:10.915468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 854/1698] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/projects => generated 365 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:10.921068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.931595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 845/1699] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:10.933724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:10.961364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66609-2109-4f2f-a383-b8d58295ad98 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.069146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031c967-8206-43e5-a44a-958b8da0e4b7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.095325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.096924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.097198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.097484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.098937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.105355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.147563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.148351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 855/1700] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:10 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:11.148889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.150735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.151773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.151923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.153173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.157812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.177776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456585dd-bb74-4e9d-9a2c-3c72930a736c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.191141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 846/1701] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:11.191354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.191354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.191354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.192949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.193666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.193924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.198802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.228081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 856/1702] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:11.229503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427111d-1a1c-4b2f-962d-9088e03f4771 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.231531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.239111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.239888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.240177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.240466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.241169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.242979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.255492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fb3ce-e855-4dc5-8488-7acefd4b05f8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.269688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.270323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.270607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.270886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.271542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.277317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.281004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 847/1703] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:11.285195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.302970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a4cb1-a0f1-4a88-9d22-bb920198b702 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.309259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 857/1704] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:11.315424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.331826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f547e-fc9e-43bb-8ee3-08b1176c28e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.332525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f547e-fc9e-43bb-8ee3-08b1176c28e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.332804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f547e-fc9e-43bb-8ee3-08b1176c28e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.333171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f547e-fc9e-43bb-8ee3-08b1176c28e9 admin admin] PATH_INFO: `/v3/projects/71d56cdb56f4479d95cec7b507d1e6a0/users/1b1bb703ce4041e4a749b5df0c2d68e4/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.338618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff3edb-c812-4940-9ca5-604323946b86 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.352678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34451-5c9d-429f-8d58-8cc9bdb70888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.353566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34451-5c9d-429f-8d58-8cc9bdb70888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.353979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34451-5c9d-429f-8d58-8cc9bdb70888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.354267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34451-5c9d-429f-8d58-8cc9bdb70888 admin admin] PATH_INFO: `/v3/projects/f16a622d05a54de8900e19602acbb09e/users/738d9dd58da64eeb8c13260747d5f9ec/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.356771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:11.365587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f547e-fc9e-43bb-8ee3-08b1176c28e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d56cdb56f4479d95cec7b507d1e6a0, user_id=1b1bb703ce4041e4a749b5df0c2d68e4, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.372803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f547e-fc9e-43bb-8ee3-08b1176c28e9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.413025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 848/1705] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:11 2022] PUT /identity/v3/projects/71d56cdb56f4479d95cec7b507d1e6a0/users/1b1bb703ce4041e4a749b5df0c2d68e4/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:11.424978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.426449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.426917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.437204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34451-5c9d-429f-8d58-8cc9bdb70888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16a622d05a54de8900e19602acbb09e, user_id=738d9dd58da64eeb8c13260747d5f9ec, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.446792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34451-5c9d-429f-8d58-8cc9bdb70888 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.506888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 858/1706] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:11 2022] PUT /identity/v3/projects/f16a622d05a54de8900e19602acbb09e/users/738d9dd58da64eeb8c13260747d5f9ec/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:11.522393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.522682 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.523033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.537331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] MFA Rules not processed for user `1b1bb703ce4041e4a749b5df0c2d68e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:11.540639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:11.556701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] MFA Rules not processed for user `738d9dd58da64eeb8c13260747d5f9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:11.557983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:11.634453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:11.718041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 849/1707] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:11.718041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.719555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:11.740107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 859/1708] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:11.741241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.840903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcb5e2-153a-42ad-a012-3f6c965cb9b6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.860652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.863153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.864588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.864906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.866349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.868695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.875204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7aa4f-82fc-402c-b322-b157b75722c7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:11.887469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:11.894947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:11.895229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:11.895518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:11.901359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:11.926968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:11.939725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:11.951873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 860/1709] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:11 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:11.951873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:11.992900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 850/1710] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:11.999383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:12.044791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e695e-9088-43fe-a2a7-e91fb61cb6d7 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:12.174043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e3662-ce27-4808-937e-6d1d4096c386 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:12.195886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0161-7969-44b6-b760-81fd90cf7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:12.196556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0161-7969-44b6-b760-81fd90cf7db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.199203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0161-7969-44b6-b760-81fd90cf7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.199763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0161-7969-44b6-b760-81fd90cf7db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.200218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b0161-7969-44b6-b760-81fd90cf7db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:12.202842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b0161-7969-44b6-b760-81fd90cf7db2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:12.210315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:12.210315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.210591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.210857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.212231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:12.213585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:12.246687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 861/1711] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:11 2022] POST /identity/v3/projects => generated 361 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:12.249506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5613c-4c33-4a2a-82cb-690a508e2f7f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.257419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5613c-4c33-4a2a-82cb-690a508e2f7f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.259352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5613c-4c33-4a2a-82cb-690a508e2f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.268428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:12.281310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 851/1712] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:11 2022] POST /identity/v3/users => generated 294 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:12.282124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:12.310159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63c929-bc8a-478a-95eb-10cf8ccebc64 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:12.320862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ebd5613c-4c33-4a2a-82cb-690a508e2f7f None None] MFA Rules not processed for user `976558170ae64a9cbbd6930e7c3b048a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:12.322638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ebd5613c-4c33-4a2a-82cb-690a508e2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:12.420753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:12.483048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:12.483709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.483988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.484284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.484926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:12.486877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:12.555807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 852/1713] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:12 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:12.556717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:12.571328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 862/1714] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:12.579156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.585929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.586388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.598888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520dde-fee5-4d3b-af62-58f29075c56e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:12.619861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:12.622790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.623867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.628480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.629448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:12.638118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:12.671681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] MFA Rules not processed for user `d045f032dbfb450aa9c087ada1637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:12.684674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:12.704736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 853/1715] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:12.706533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:12.737687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2650d-9b5f-46c0-9da6-968298c73d27 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:12.758616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:12.759838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.760501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.761126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.762455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:12.770924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:12.832701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 854/1716] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:12.834507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:12.896343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7b3b6-d111-4432-98fc-81670d629803 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:12.915762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:12.916591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:12.916988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:12.917350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:12.919005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:12.921664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:12.953562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 855/1717] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:12.968593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:12.995229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af48-7f59-4980-a75d-8599b6149852 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:13.001470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 863/1718] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:13.010435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:13.025092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:13.025924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.026914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.026914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] PATH_INFO: `/v3/projects/f90b7d6159e74c5eab4b10f6c6a49ba3/users/7d7db0886ab24ad199b16af5710e1961/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.036822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5407-c48e-476f-9c10-c2c3a238aea3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:13.051370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:13.052253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.053777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.053777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] PATH_INFO: `/v3/projects/c4b3df653cbb45ad9e02478c404d798b/users/6310e1a32f10439f9cdcbfbb57e82134/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.062675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90b7d6159e74c5eab4b10f6c6a49ba3, user_id=7d7db0886ab24ad199b16af5710e1961, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:13.067112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:13.090530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b3df653cbb45ad9e02478c404d798b, user_id=6310e1a32f10439f9cdcbfbb57e82134, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:13.103622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:13.112713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 856/1719] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:12 2022] PUT /identity/v3/projects/f90b7d6159e74c5eab4b10f6c6a49ba3/users/7d7db0886ab24ad199b16af5710e1961/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:13.115437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:13.122935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:13.145823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 864/1720] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:13 2022] PUT /identity/v3/projects/c4b3df653cbb45ad9e02478c404d798b/users/6310e1a32f10439f9cdcbfbb57e82134/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:13.158252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:13.369859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a2e49-4425-4c37-b51a-3887146257d2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:13.383202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fb26d-bac6-4f89-a485-957fbe79278a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:13.393388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1332d4-44ce-4593-adc6-5daed1a5915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:13.393873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1332d4-44ce-4593-adc6-5daed1a5915a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.394609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1332d4-44ce-4593-adc6-5daed1a5915a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.394609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1332d4-44ce-4593-adc6-5daed1a5915a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.395353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1332d4-44ce-4593-adc6-5daed1a5915a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:13.398175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1332d4-44ce-4593-adc6-5daed1a5915a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:13.429872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 865/1721] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:13.441433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.441433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.441433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.502626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:13.547732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] MFA Rules not processed for user `6310e1a32f10439f9cdcbfbb57e82134`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:13.549059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:13.635606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:13.660942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:13.662883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.664689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.666154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.803678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 866/1722] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:13.815078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:13.844994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d9823-94db-474b-8f17-f3e43a689257 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:13.859266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8471137-d602-4eb0-aa9c-4958b35f17ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:13.863301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8471137-d602-4eb0-aa9c-4958b35f17ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.863301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8471137-d602-4eb0-aa9c-4958b35f17ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.863301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8471137-d602-4eb0-aa9c-4958b35f17ca admin admin] PATH_INFO: `/v3/projects/f90b7d6159e74c5eab4b10f6c6a49ba3/users/7d7db0886ab24ad199b16af5710e1961/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.869305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8471137-d602-4eb0-aa9c-4958b35f17ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90b7d6159e74c5eab4b10f6c6a49ba3, user_id=7d7db0886ab24ad199b16af5710e1961, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:13.876590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8471137-d602-4eb0-aa9c-4958b35f17ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:13.900964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:13.902714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:13.931064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 857/1723] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:13.936380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:13.943862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 867/1724] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:13 2022] PUT /identity/v3/projects/f90b7d6159e74c5eab4b10f6c6a49ba3/users/7d7db0886ab24ad199b16af5710e1961/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:13.955345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:13.955692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:13.956080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:13.973769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8a7a0-8019-498c-8ea4-74ab03c0b06d service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:14.074125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] MFA Rules not processed for user `7d7db0886ab24ad199b16af5710e1961`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:14.077717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:14.239877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:14.250030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:14.250937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:14.252373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:14.253137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:14.258975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:14.356107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 858/1725] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:13 2022] POST /identity/v3/projects => generated 385 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:14.361096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:14.399302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 868/1726] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:14.414455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:14.419668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:14.573081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:14.618040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb3392-40b3-45f2-9843-4300f33b75ba admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:14.635403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:14.637903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ae182-1ede-4683-b47c-b2f6247701af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:14.641612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:14.641769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:14.651461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:14.659862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:14.668339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:14.756248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 859/1727] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:14 2022] POST /identity/v3/users => generated 299 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:14.757753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:14.776931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:14.778149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:14.779043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411dcec-888f-477c-b564-d82746e9d039 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:14.779603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:14.780679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:14.794247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:14.809704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:14.811116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:14.811451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:14.811705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:14.812485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:14.814120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:14.859651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 860/1728] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:14.861921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:14.901420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154686-2f95-4f07-af0c-659351fcd770 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:14.926883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:14.927561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:14.927839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:14.928112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] PATH_INFO: `/v3/projects/78173d6b8dd44eb29b41afda05abf06b/users/a0be7773ede24ed18f4191fd18651399/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:14.956065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:14.958193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:14.973100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78173d6b8dd44eb29b41afda05abf06b, user_id=a0be7773ede24ed18f4191fd18651399, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:14.980558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:15.013176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 861/1729] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:14 2022] PUT /identity/v3/projects/78173d6b8dd44eb29b41afda05abf06b/users/a0be7773ede24ed18f4191fd18651399/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:15.018125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:15.061867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 869/1730] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:15.218453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f89c3-96b8-4c69-82d1-1034bfa60f92 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:15.230291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:15.231129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:15.232005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:15.234263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:15.235794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:15.240046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:15.278768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:15.297860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 862/1731] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:15.298405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc352e-e45a-4312-bf9a-131c67a40a34 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:15.302058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:15.341390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e2300-7583-4905-bf69-5e100fa86f56 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:15.369078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501661d9-60ed-4166-bf89-c3440b9c0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:15.372958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501661d9-60ed-4166-bf89-c3440b9c0585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:15.377975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501661d9-60ed-4166-bf89-c3440b9c0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:15.379764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501661d9-60ed-4166-bf89-c3440b9c0585 admin admin] PATH_INFO: `/v3/projects/78173d6b8dd44eb29b41afda05abf06b/users/a0be7773ede24ed18f4191fd18651399/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:15.387386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501661d9-60ed-4166-bf89-c3440b9c0585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78173d6b8dd44eb29b41afda05abf06b, user_id=a0be7773ede24ed18f4191fd18651399, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:15.405596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501661d9-60ed-4166-bf89-c3440b9c0585 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:15.456123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 863/1732] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:15 2022] PUT /identity/v3/projects/78173d6b8dd44eb29b41afda05abf06b/users/a0be7773ede24ed18f4191fd18651399/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:15.469402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:15.469744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:15.470193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:15.544390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af256cc5-4ed2-4c53-a81b-d2c5e84f01fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:15.546196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af256cc5-4ed2-4c53-a81b-d2c5e84f01fd service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:15.546762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af256cc5-4ed2-4c53-a81b-d2c5e84f01fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:15.549727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af256cc5-4ed2-4c53-a81b-d2c5e84f01fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:15.593996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] MFA Rules not processed for user `a0be7773ede24ed18f4191fd18651399`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:15.598088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:15.819751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af256cc5-4ed2-4c53-a81b-d2c5e84f01fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:15.819751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af256cc5-4ed2-4c53-a81b-d2c5e84f01fd service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:15.889512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 870/1733] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:15.904694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 864/1734] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:16.203286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:16.203969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:16.204519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:16.334198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:16.371102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:16.387237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:16.390313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:16.555240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47a52-5718-4380-841a-e2aa31cc207a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:16.563473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:16.564111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:16.564417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:16.564692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:16.568617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:16.590645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:16.596169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:16.636439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 865/1735] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:16.637819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:16.642292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:16.691552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 871/1736] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:16.821794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d890a1-5945-44da-b275-19c797d1bc68 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:16.847465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:16.848130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:16.848446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:16.848720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] PATH_INFO: `/v3/users/0e6c8cee157e45869cc079bb211513b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:16.852278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e6c8cee157e45869cc079bb211513b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:16.854398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:16.958039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:16.959656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.022985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e6c8cee157e45869cc079bb211513b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:17.063924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e6c8cee157e45869cc079bb211513b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:17.084783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e6c8cee157e45869cc079bb211513b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:17.086582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e6c8cee157e45869cc079bb211513b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:17.087472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d725180b-8152-4f6c-b604-b221d9b94275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e6c8cee157e45869cc079bb211513b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:17.115892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 866/1737] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:16 2022] DELETE /identity/v3/users/0e6c8cee157e45869cc079bb211513b5 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:17.120568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a0e7-652a-416e-87f8-de131cc4ea67 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:17.121054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a0e7-652a-416e-87f8-de131cc4ea67 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:17.121471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a0e7-652a-416e-87f8-de131cc4ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:17.159652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-12c7a0e7-652a-416e-87f8-de131cc4ea67 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:17.161187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-12c7a0e7-652a-416e-87f8-de131cc4ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.168606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d8bc5-5910-4b63-b87b-8e9bd28b3c6d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:17.214397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:17.330757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 867/1738] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:17.332475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:17.332921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:17.333333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:17.369197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:17.370263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:17.370732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:17.371230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:17.385152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.455847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] MFA Rules not processed for user `dae3738120794f9fa52c8054e3c462ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:17.461121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.531938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:17.549823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:17.551483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:17.586251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 872/1739] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:16:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 11:16:17.587111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:17.597125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.677560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 868/1740] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:17.690661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:17.691981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.835443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cb179-d4dc-4751-9c52-d23687861b37 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:17.860869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:17.861511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:17.861805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:17.862060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:17.905355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:17.906825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:17.922130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbaff8-712e-47a0-bcce-4acd3a70d8fd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:17.944813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 873/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:17.945717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:17.947655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da20c50-73f7-4545-89c0-c961cd4cb5dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:17.951243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da20c50-73f7-4545-89c0-c961cd4cb5dd service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:17.952364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da20c50-73f7-4545-89c0-c961cd4cb5dd service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:17.953089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da20c50-73f7-4545-89c0-c961cd4cb5dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:17.964585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:17.978150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da20c50-73f7-4545-89c0-c961cd4cb5dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:17.980583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da20c50-73f7-4545-89c0-c961cd4cb5dd service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:17.997654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 869/1742] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:18.246478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6a565-b131-4f0c-8d39-7111a7435694 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:18.269497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:18.270138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:18.270424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:18.270698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:18.292271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:18.572508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:18.573979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:18.602186 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 874/1743] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:19.575873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dbf59-375e-403a-9839-a81a668a7b1a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:19.576596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dbf59-375e-403a-9839-a81a668a7b1a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:19.577012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dbf59-375e-403a-9839-a81a668a7b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:19.709964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7c2dbf59-375e-403a-9839-a81a668a7b1a None None] MFA Rules not processed for user `faf58e78faae461fa0e7b75eb30c6c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:19.718693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7c2dbf59-375e-403a-9839-a81a668a7b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:19.720170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:19.721741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:19.749462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10e60-2957-47e0-b698-b48a7bfc768c service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:19.764091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:19.765637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:19.766240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:19.766670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] PATH_INFO: `/v3/projects/f1e1b718c6b944cfb3f5122dc4d25501` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:19.768957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e1b718c6b944cfb3f5122dc4d25501)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:19.771190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:19.886849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:19.950036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:19.954529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e1b718c6b944cfb3f5122dc4d25501', 'request_id': 'req-8aefca36-b3ba-4a31-adfc-999aa2933c8e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:19.972311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 870/1744] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:19.984212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:19.984455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e1b718c6b944cfb3f5122dc4d25501', 'request_id': 'req-8aefca36-b3ba-4a31-adfc-999aa2933c8e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:19.984562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:19.984834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:19.994702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e1b718c6b944cfb3f5122dc4d25501', 'request_id': 'req-8aefca36-b3ba-4a31-adfc-999aa2933c8e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.033946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] The token cache is being invalidate because project f1e1b718c6b944cfb3f5122dc4d25501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:20.034457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.035308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aefca36-b3ba-4a31-adfc-999aa2933c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.037126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 875/1745] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:19 2022] DELETE /identity/v3/projects/f1e1b718c6b944cfb3f5122dc4d25501 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:20.042124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523b2c6-1655-465e-acb2-6fd1d907f7f9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:20.042480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523b2c6-1655-465e-acb2-6fd1d907f7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:20.042674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523b2c6-1655-465e-acb2-6fd1d907f7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:20.105712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] MFA Rules not processed for user `faf58e78faae461fa0e7b75eb30c6c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:20.108970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:20.121844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7523b2c6-1655-465e-acb2-6fd1d907f7f9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:20.125929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7523b2c6-1655-465e-acb2-6fd1d907f7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:20.267849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 871/1746] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:20.275695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:20.277699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:20.361110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 876/1747] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:20.370244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:20.370540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:20.370883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:20.485143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:20.486390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:20.530327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 877/1748] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:20.531443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:20.566152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1f699-1ce3-4f5a-bbd5-4348984ef037 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:20.588288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:20.589789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:20.591169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:20.595031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:20.595937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:20.596463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:20.598365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:20.633372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813ad81-a24a-4424-a993-9002cc45338b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:20.649280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 878/1749] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:20.651424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:20.663334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:20.671814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:20.672225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:20.672564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:20.672843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:20.683552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:20.727448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e54d-3aba-4fb7-b2e7-67aff81f0c79 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:20.740712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:20.746319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:20.757700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:20.758688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:20.761312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:20.761640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] PATH_INFO: `/v3/users/738d9dd58da64eeb8c13260747d5f9ec` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:20.769036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=738d9dd58da64eeb8c13260747d5f9ec)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:20.776984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:20.785150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 872/1750] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:20.791461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:20.825220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd8362-ce6e-441f-ab3e-d75c4cfa67e8 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:20.838012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:20.839085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:20.839575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:20.840902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:20.841863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:20.844105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:20.927993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738d9dd58da64eeb8c13260747d5f9ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.927993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738d9dd58da64eeb8c13260747d5f9ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.928400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738d9dd58da64eeb8c13260747d5f9ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.969533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738d9dd58da64eeb8c13260747d5f9ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.989693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 873/1751] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:20 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:20.991049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b802b8b-c635-4a97-9ebe-33fa672f86ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738d9dd58da64eeb8c13260747d5f9ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:20.992411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:21.006438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:21.036117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 879/1752] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:20 2022] DELETE /identity/v3/users/738d9dd58da64eeb8c13260747d5f9ec => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:21.041384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7661c-13fc-4b8c-945b-fcec9f19d89a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.041663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7661c-13fc-4b8c-945b-fcec9f19d89a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.042090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7661c-13fc-4b8c-945b-fcec9f19d89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.080745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-34a7661c-13fc-4b8c-945b-fcec9f19d89a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:21.094179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-34a7661c-13fc-4b8c-945b-fcec9f19d89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:21.145185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:21.177389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:21.222822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18499d-f9fa-45d4-87c7-bb017d57f709 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:21.231651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:21.233356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.234038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.234571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.238467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:21.239201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 880/1753] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:21.239201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.239201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.239201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.240457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:21.321698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 874/1754] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:20 2022] POST /identity/v3/users => generated 295 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:21.323134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:21.328075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:21.329260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] MFA Rules not processed for user `d045f032dbfb450aa9c087ada1637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:21.332069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:21.509253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 881/1755] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:21.522079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:21.547562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bb134-274c-42ce-a664-ba1d7c0dc6dc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:21.550437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e94ff4-5886-4cc8-91df-83c75ca4b45b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:21.565315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:21.566793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.567087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.567361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.579987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:21.580985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.581424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.581841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.582713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:21.588306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:21.597445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:21.598909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:21.619428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 882/1756] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:21.625264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:21.650014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d2e8e-715c-406a-b969-8bfdef7a2ec2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:21.650712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 875/1757] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:21.676181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76d6e6-1e75-4508-8a54-44605c1d2ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:21.676887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76d6e6-1e75-4508-8a54-44605c1d2ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.677183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76d6e6-1e75-4508-8a54-44605c1d2ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.677435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76d6e6-1e75-4508-8a54-44605c1d2ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.678083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76d6e6-1e75-4508-8a54-44605c1d2ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:21.682183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76d6e6-1e75-4508-8a54-44605c1d2ca3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:21.711657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:21.720037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 883/1758] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:21.727825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bfdfd-17ae-41f5-96af-5e8d3db56212 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:21.744344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fa92c-79c5-48db-b9a6-25d90eea402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:21.745021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fa92c-79c5-48db-b9a6-25d90eea402d admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.745318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fa92c-79c5-48db-b9a6-25d90eea402d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.745589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fa92c-79c5-48db-b9a6-25d90eea402d admin admin] PATH_INFO: `/v3/projects/6f7a76d1ff164f30a538a0de5e4dadfd/users/e8ee4bf59ba34bfaa1d54806bdf7435a/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.800374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fa92c-79c5-48db-b9a6-25d90eea402d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7a76d1ff164f30a538a0de5e4dadfd, user_id=e8ee4bf59ba34bfaa1d54806bdf7435a, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:21.810885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fa92c-79c5-48db-b9a6-25d90eea402d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:21.871807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.873042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.873493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:21.876987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 876/1759] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:21 2022] PUT /identity/v3/projects/6f7a76d1ff164f30a538a0de5e4dadfd/users/e8ee4bf59ba34bfaa1d54806bdf7435a/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:21.909533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] MFA Rules not processed for user `e8ee4bf59ba34bfaa1d54806bdf7435a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:21.911252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:21.984023 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:21.988617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:21.988950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:22.042367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:22.043591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:22.185989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 884/1760] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:22.190925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:22.269134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 877/1761] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Aug 18 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:22.324573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:22.413907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab854e-c353-43af-93dd-3aa092fc2e13 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:22.428909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:22.433638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:22.434016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:22.434852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:22.435605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:22.447499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:22.507829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:22.511852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:22.538755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 885/1762] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:22 2022] POST /identity/v3/projects => generated 385 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:22.545454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:22.558827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806cd41-dced-4ce2-ae13-8239b10f7f7a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:22.727797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf971ea-a0de-4570-b1cc-0e01ff24b877 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:22.741946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:22.743497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:22.743862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:22.744322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:22.745183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:22.747432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:22.761260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:22.762109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:22.762546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:22.762984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:22.782753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:22.784456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:22.796675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 886/1763] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:22 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:22.810291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 878/1764] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:22.811324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:22.812380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:22.819840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:22.839901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fab8c-1e6d-4444-b1a3-28cc61baf616 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:22.853338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:22.854152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:22.854534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:22.854920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:22.855795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:22.859701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:22.878928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7769f-042a-42da-bcd6-e73f26f5b5fe admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:22.880061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:22.896626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 879/1765] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:22.898743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:22.902001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:22.904457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:22.904953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:22.907660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] PATH_INFO: `/v3/projects/f16a622d05a54de8900e19602acbb09e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:22.912860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16a622d05a54de8900e19602acbb09e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:22.927460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:22.935158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea0726-156f-4927-94dd-2f0ad1bc6d56 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:22.971161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:22.972111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:22.972546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:22.972929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] PATH_INFO: `/v3/projects/7fb35322c5f24c4188ee1bf278ffdfa5/users/163d4e49af61408690bb79fab4f981f9/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:23.081983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb35322c5f24c4188ee1bf278ffdfa5, user_id=163d4e49af61408690bb79fab4f981f9, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:23.084973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:23.210145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16a622d05a54de8900e19602acbb09e', 'request_id': 'req-c36ee549-51a9-4104-9c13-cad03d697328'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:23.214368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 880/1766] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:22 2022] PUT /identity/v3/projects/7fb35322c5f24c4188ee1bf278ffdfa5/users/163d4e49af61408690bb79fab4f981f9/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:23.222903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:23.230741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16a622d05a54de8900e19602acbb09e', 'request_id': 'req-c36ee549-51a9-4104-9c13-cad03d697328'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:23.247324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16a622d05a54de8900e19602acbb09e', 'request_id': 'req-c36ee549-51a9-4104-9c13-cad03d697328'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:23.280886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] The token cache is being invalidate because project f16a622d05a54de8900e19602acbb09e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:23.281558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:23.283457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:23.285766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 887/1767] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:22 2022] DELETE /identity/v3/projects/f16a622d05a54de8900e19602acbb09e => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:23.291032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:23.302076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:23.531509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:23.596302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ee549-51a9-4104-9c13-cad03d697328 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:23.723248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b587fc-0c93-4a19-9ff9-a439495dc086 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:23.756956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:23.758701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:23.760880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:23.761213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:23.765507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:23.771614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:23.814058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 881/1768] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 593 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:23.815915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:23.821421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:23.885312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff78747-43c2-46ed-997c-fc960ebdc6bc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:23.893189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5e535-9f86-4266-b431-42712259a24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:23.894345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5e535-9f86-4266-b431-42712259a24e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:23.894892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5e535-9f86-4266-b431-42712259a24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:23.895423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5e535-9f86-4266-b431-42712259a24e admin admin] PATH_INFO: `/v3/projects/7fb35322c5f24c4188ee1bf278ffdfa5/users/163d4e49af61408690bb79fab4f981f9/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:23.898243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5e535-9f86-4266-b431-42712259a24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb35322c5f24c4188ee1bf278ffdfa5, user_id=163d4e49af61408690bb79fab4f981f9, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:23.902373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5e535-9f86-4266-b431-42712259a24e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:23.952788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 882/1769] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:23 2022] PUT /identity/v3/projects/7fb35322c5f24c4188ee1bf278ffdfa5/users/163d4e49af61408690bb79fab4f981f9/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:23.984075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:23.987943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:23.988700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:24.010256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:24.016571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:24.018713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:24.019179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] PATH_INFO: `/v3/users/7d7db0886ab24ad199b16af5710e1961` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:24.021197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7db0886ab24ad199b16af5710e1961)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:24.036885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:24.141997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] MFA Rules not processed for user `163d4e49af61408690bb79fab4f981f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:24.146144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:24.308112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7db0886ab24ad199b16af5710e1961'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:24.310381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7db0886ab24ad199b16af5710e1961'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:24.312650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7db0886ab24ad199b16af5710e1961'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:24.407033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 883/1770] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:24.408244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7db0886ab24ad199b16af5710e1961'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:24.421892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:24.425697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:24.432633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-925cb43b-edad-46e2-8d63-d5df3cab8014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7db0886ab24ad199b16af5710e1961'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:24.487414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 888/1771] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:23 2022] DELETE /identity/v3/users/7d7db0886ab24ad199b16af5710e1961 => generated 0 bytes in 1196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:24.545418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:24.566443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe6ccf-11b9-4826-8aad-416e9af72b49 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:24.780277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db12344-17d7-4a22-a793-4b9afafe4a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:24.781874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db12344-17d7-4a22-a793-4b9afafe4a39 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:24.782622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db12344-17d7-4a22-a793-4b9afafe4a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:24.783347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db12344-17d7-4a22-a793-4b9afafe4a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:24.969016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db12344-17d7-4a22-a793-4b9afafe4a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:24.971504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db12344-17d7-4a22-a793-4b9afafe4a39 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:25.053321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 884/1772] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:25.213760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a2a47-f5cf-4c66-b40a-767df89b483f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:25.214301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a2a47-f5cf-4c66-b40a-767df89b483f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:25.214686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a2a47-f5cf-4c66-b40a-767df89b483f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:25.218017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 889/1773] 10.222.0.16 () {54 vars in 912 bytes} [Thu Aug 18 11:16:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:16:25.229477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:25.229809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:25.230103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:25.339938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:25.361058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:25.448424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a2a47-f5cf-4c66-b40a-767df89b483f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:25.452530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-557a2a47-f5cf-4c66-b40a-767df89b483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:25.676218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 885/1774] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:16:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:25.801413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a2a47-f5cf-4c66-b40a-767df89b483f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:25.819316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:25.819976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:25.820294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:25.820558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] PATH_INFO: `/v3/users/e8ee4bf59ba34bfaa1d54806bdf7435a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:25.827512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ee4bf59ba34bfaa1d54806bdf7435a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:25.832067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:25.850231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:26.064694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ee4bf59ba34bfaa1d54806bdf7435a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.069835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ee4bf59ba34bfaa1d54806bdf7435a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.072665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ee4bf59ba34bfaa1d54806bdf7435a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.162079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ee4bf59ba34bfaa1d54806bdf7435a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.188769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-efca1df6-b7c8-4251-84d8-de83a23394c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ee4bf59ba34bfaa1d54806bdf7435a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.210976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:26.221033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:26.234608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 890/1775] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:25 2022] DELETE /identity/v3/users/e8ee4bf59ba34bfaa1d54806bdf7435a => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:26.237515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:26.240930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:26.242000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:26.266223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fb17e-f598-41c7-a99b-8f632ca30433 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:26.337300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:26.341300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:26.501953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:26.514625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:26.518211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:26.518651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:26.526385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:26.576130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:26.578608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:26.618537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 886/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:26.630880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:26.634861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:26.643508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:26.715428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 891/1777] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:26.729437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:26.731987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:26.760100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1c463-8f7a-433d-8a3b-ffb2aeeb6463 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:26.761470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded9356-aa2e-42e5-9ed9-e8df1bdab10b service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:26.769251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab87667-9f48-490a-9615-feba21bb8f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:26.770328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87667-9f48-490a-9615-feba21bb8f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:26.771347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87667-9f48-490a-9615-feba21bb8f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:26.772535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:26.773397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:26.773710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87667-9f48-490a-9615-feba21bb8f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:26.775318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:26.775760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] PATH_INFO: `/v3/projects/f90b7d6159e74c5eab4b10f6c6a49ba3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:26.778533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90b7d6159e74c5eab4b10f6c6a49ba3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:26.783031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:26.800533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab87667-9f48-490a-9615-feba21bb8f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:26.802320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab87667-9f48-490a-9615-feba21bb8f5f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:26.831837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 892/1778] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:26.917178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90b7d6159e74c5eab4b10f6c6a49ba3', 'request_id': 'req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.932496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90b7d6159e74c5eab4b10f6c6a49ba3', 'request_id': 'req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:26.956956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90b7d6159e74c5eab4b10f6c6a49ba3', 'request_id': 'req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:27.005834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] The token cache is being invalidate because project f90b7d6159e74c5eab4b10f6c6a49ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:27.006091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:27.008775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5fc7eaf7-fbad-4ee1-887d-852fa6efd49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:27.019202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 887/1779] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:26 2022] DELETE /identity/v3/projects/f90b7d6159e74c5eab4b10f6c6a49ba3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:27.021006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:27.021476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:27.021946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:27.079161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:27.088056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:27.279311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 893/1780] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:27.281695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:27.282479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:27.282923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:27.310305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:27.314393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:27.333167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:27.348663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 888/1781] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:27.351231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3c952-9c08-41cf-ba0f-68029fd04f7a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:27.372797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:27.373808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:27.374278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:27.374717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:27.375624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:27.379027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:27.381794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:27.416932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:27.439468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 894/1782] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:27.446134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:27.512302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04da8b-ed8f-4cbd-ad45-a393c333c9ab None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:27.543852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:27.544545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:27.544834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:27.545083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:27.546101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:27.550083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:27.619359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 889/1783] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:27 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:27.624475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:27.648272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2685-2d4c-4de8-854e-a731d69e8ef1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:27.703376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:27.705677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:27.881156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:27.882138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:27.882138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:27.882351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:27.883632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:27.886037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:27.930509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05244e-2713-4efc-aaab-470c1d978509 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:27.939238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 895/1784] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:27 2022] POST /identity/v3/users => generated 291 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:27.962563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:27.987567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:27.991910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:27.993421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:27.994748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] PATH_INFO: `/v3/projects/6f7a76d1ff164f30a538a0de5e4dadfd` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:28.010366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7a76d1ff164f30a538a0de5e4dadfd)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:28.010827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e81f-1c41-4ca4-9833-b93f5ee6f923 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:28.038977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:28.039609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:28.039886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:28.040286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:28.040844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:28.042490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:28.042625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:28.077801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:28.119337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 896/1785] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:28.124016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:28.155368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393594e2-78aa-4978-95a1-dbc12275b679 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:28.178366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7a76d1ff164f30a538a0de5e4dadfd', 'request_id': 'req-37408b3f-6c76-4170-a589-f68462acbf05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.201535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7a76d1ff164f30a538a0de5e4dadfd', 'request_id': 'req-37408b3f-6c76-4170-a589-f68462acbf05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.226607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7a76d1ff164f30a538a0de5e4dadfd', 'request_id': 'req-37408b3f-6c76-4170-a589-f68462acbf05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.245189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] The token cache is being invalidate because project 6f7a76d1ff164f30a538a0de5e4dadfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:28.245504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.246516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.249545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 890/1786] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:27 2022] DELETE /identity/v3/projects/6f7a76d1ff164f30a538a0de5e4dadfd => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:28.253475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:28.255572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:28.375696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:28.376391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37408b3f-6c76-4170-a589-f68462acbf05 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:28.377836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:28.378456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:28.379067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:28.380124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:28.382542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:28.383490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:28.384148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:28.384466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:28.384744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] PATH_INFO: `/v3/users/163d4e49af61408690bb79fab4f981f9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:28.386723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=163d4e49af61408690bb79fab4f981f9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:28.389386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:28.397949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 897/1787] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:28.402453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:28.407318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:28.445014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e263617-b747-4871-84ae-e6649be74b72 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:28.453673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888737b-efe2-4e4f-8aba-433bcf5f3255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:28.454509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888737b-efe2-4e4f-8aba-433bcf5f3255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:28.454869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888737b-efe2-4e4f-8aba-433bcf5f3255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:28.455256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888737b-efe2-4e4f-8aba-433bcf5f3255 admin admin] PATH_INFO: `/v3/projects/4e2523f00ead4f4995530e0f9b77356c/users/98cbfae893e7462899c4f9e385c7f895/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:28.487999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888737b-efe2-4e4f-8aba-433bcf5f3255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2523f00ead4f4995530e0f9b77356c, user_id=98cbfae893e7462899c4f9e385c7f895, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:28.493393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888737b-efe2-4e4f-8aba-433bcf5f3255 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:28.497213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163d4e49af61408690bb79fab4f981f9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.538135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 898/1788] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:28 2022] PUT /identity/v3/projects/4e2523f00ead4f4995530e0f9b77356c/users/98cbfae893e7462899c4f9e385c7f895/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:28.550990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:28.550990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:28.551255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:28.559426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163d4e49af61408690bb79fab4f981f9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.583815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163d4e49af61408690bb79fab4f981f9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.585082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163d4e49af61408690bb79fab4f981f9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.586115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163d4e49af61408690bb79fab4f981f9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:28.624588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] MFA Rules not processed for user `98cbfae893e7462899c4f9e385c7f895`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:28.626193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:28.636534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 891/1789] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:28 2022] DELETE /identity/v3/users/163d4e49af61408690bb79fab4f981f9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:28.755755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:28.811292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 899/1790] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:28.816259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:28.817858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:28.842632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:29.004376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f2f1-9d3f-446c-8a3d-4a6176324669 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:29.031375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69d97f-70bf-49e7-aadf-ac383cd9c1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:29.031375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69d97f-70bf-49e7-aadf-ac383cd9c1dd admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.031375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69d97f-70bf-49e7-aadf-ac383cd9c1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.031375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69d97f-70bf-49e7-aadf-ac383cd9c1dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.031784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69d97f-70bf-49e7-aadf-ac383cd9c1dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:29.038781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69d97f-70bf-49e7-aadf-ac383cd9c1dd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:29.106464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 892/1791] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:28 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:29.110863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:29.207252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.207604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.208005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.251398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b05190-64a9-41cd-afe8-62c0b42ca6e0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:29.258168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] MFA Rules not processed for user `dae3738120794f9fa52c8054e3c462ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:29.259518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:29.439460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:29.440869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.441972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.444725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.445437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:29.446564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 893/1792] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:29.453933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:29.512158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 900/1793] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:29 2022] POST /identity/v3/users => generated 297 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:29.520253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:29.541483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318efe6-5f46-4ee2-b2b3-c9a572ea0153 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:29.556005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:29.556820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.557597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.558154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.559162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:29.565587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:29.597720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 894/1794] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:29.602208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:29.621844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80814f-2029-4d2c-8734-4cfb4d67ef9a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:29.636532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:29.637357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.637742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.638115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] PATH_INFO: `/v3/projects/57aea120f26c4ec0803a993cded94271/users/55ac62ee356941fb9c54c652ee2a5b34/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.679853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] RBAC: Authorizing `identity:create_grant(project_id=57aea120f26c4ec0803a993cded94271, user_id=55ac62ee356941fb9c54c652ee2a5b34, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:29.690962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:29.719935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 901/1795] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:29 2022] PUT /identity/v3/projects/57aea120f26c4ec0803a993cded94271/users/55ac62ee356941fb9c54c652ee2a5b34/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:29.725746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:29.746341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7b7e6-78b8-43fc-acbb-741eb9338303 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:29.916038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3356cbd-0d69-4223-872b-5f55ecf5b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:29.916674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3356cbd-0d69-4223-872b-5f55ecf5b55c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.916956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3356cbd-0d69-4223-872b-5f55ecf5b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.917238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3356cbd-0d69-4223-872b-5f55ecf5b55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.917889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3356cbd-0d69-4223-872b-5f55ecf5b55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:29.919510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3356cbd-0d69-4223-872b-5f55ecf5b55c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:29.938458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 895/1796] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:29.939798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:29.973661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e381d36-9e8d-473b-a4da-d7d7c8fdfbad admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:29.980471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4337f9-0193-49b1-942b-63462623ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:29.981217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4337f9-0193-49b1-942b-63462623ed0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:29.981503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4337f9-0193-49b1-942b-63462623ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:29.981779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4337f9-0193-49b1-942b-63462623ed0a admin admin] PATH_INFO: `/v3/projects/57aea120f26c4ec0803a993cded94271/users/55ac62ee356941fb9c54c652ee2a5b34/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:29.983890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4337f9-0193-49b1-942b-63462623ed0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57aea120f26c4ec0803a993cded94271, user_id=55ac62ee356941fb9c54c652ee2a5b34, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:29.992047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4337f9-0193-49b1-942b-63462623ed0a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:30.026099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 902/1797] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:29 2022] PUT /identity/v3/projects/57aea120f26c4ec0803a993cded94271/users/55ac62ee356941fb9c54c652ee2a5b34/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:30.036468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ebc94-5a5a-487f-87ca-8a6afc3959a8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:30.036915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ebc94-5a5a-487f-87ca-8a6afc3959a8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:30.037197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ebc94-5a5a-487f-87ca-8a6afc3959a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:30.096038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2f3ebc94-5a5a-487f-87ca-8a6afc3959a8 None None] MFA Rules not processed for user `55ac62ee356941fb9c54c652ee2a5b34`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:30.097432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f3ebc94-5a5a-487f-87ca-8a6afc3959a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:30.128690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:30.214644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbcf93-f205-4ad3-bde4-4bd36c47f465 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:30.215193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbcf93-f205-4ad3-bde4-4bd36c47f465 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:30.215467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbcf93-f205-4ad3-bde4-4bd36c47f465 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:30.218142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 903/1798] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 11:16:30.231845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 896/1799] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:30.246161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbcf93-f205-4ad3-bde4-4bd36c47f465 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:30.268734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1edbcf93-f205-4ad3-bde4-4bd36c47f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:30.363038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:30.363736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:30.363736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:30.414758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:30.416184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:30.441777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:30.487979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbcf93-f205-4ad3-bde4-4bd36c47f465 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:30.501007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:30.502797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:30.503395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:30.503888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:30.526731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:30.533058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:30.597522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 904/1800] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:30.599025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:30.604046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 897/1801] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:16:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:30.606411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:30.803482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:30.811476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdbc7c-b169-4f45-937f-108556129ded service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:30.811699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5303be-756e-4eb0-85c0-fe2480811ff9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:30.823524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:30.824215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:30.824494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:30.825027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] PATH_INFO: `/v3/projects/7fb35322c5f24c4188ee1bf278ffdfa5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:30.828123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb35322c5f24c4188ee1bf278ffdfa5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:30.833475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:30.835880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:30.836554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:30.838239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:30.838595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:30.955263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb35322c5f24c4188ee1bf278ffdfa5', 'request_id': 'req-5e0933d4-611b-4d3f-9c58-b692e867c306'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:30.969425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb35322c5f24c4188ee1bf278ffdfa5', 'request_id': 'req-5e0933d4-611b-4d3f-9c58-b692e867c306'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:30.980952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb35322c5f24c4188ee1bf278ffdfa5', 'request_id': 'req-5e0933d4-611b-4d3f-9c58-b692e867c306'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:31.014377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:31.016589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:31.020153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] The token cache is being invalidate because project 7fb35322c5f24c4188ee1bf278ffdfa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:31.020764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:31.021966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5e0933d4-611b-4d3f-9c58-b692e867c306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:31.040468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 905/1802] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:30 2022] DELETE /identity/v3/projects/7fb35322c5f24c4188ee1bf278ffdfa5 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:31.046051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 898/1803] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:31.049428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9b13-a1d5-4ff0-bf9b-2b1f7e0a0222 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:31.049956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9b13-a1d5-4ff0-bf9b-2b1f7e0a0222 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:31.050402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9b13-a1d5-4ff0-bf9b-2b1f7e0a0222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:31.093002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2cfe9b13-a1d5-4ff0-bf9b-2b1f7e0a0222 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:31.094423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2cfe9b13-a1d5-4ff0-bf9b-2b1f7e0a0222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:31.134659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:31.139802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:31.196254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b5027-9f05-4442-a599-9a21a7ad1b29 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:31.323431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:31.349624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 906/1804] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:31.360763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:31.366946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:31.367868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:31.411341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:31.413519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:31.413519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:31.413519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:31.423093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:31.457172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:31.458527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:31.484801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:31.486569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:31.504532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 907/1805] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:31.506240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:31.510384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 899/1806] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:31.517915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:31.538053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560a6c-7ab1-4165-bbb8-27a5387a15f9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:31.568567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:31.569210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:31.569486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:31.569756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:31.570429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:31.570733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:31.572513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:31.614330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 908/1807] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:16:31.615726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:31.633755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96279d08-1d68-449a-987d-5532eaa8cbe6 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:31.642476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:31.645573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:31.646067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:31.646648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:31.651444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:31.655787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:31.710933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 900/1808] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:31 2022] POST /identity/v3/projects => generated 367 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:31.712294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:31.735946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52383b-f55a-40a9-8f3f-740a8376a94f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:32.142179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:32.143285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.143808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.144343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.145244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:32.152776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:32.206386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 909/1809] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:31 2022] POST /identity/v3/users => generated 284 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:32.207705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:32.234235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d33a0-a63b-4067-8b4f-1233a2abfd0d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:32.253968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:32.257332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.257716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.258834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.259597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:32.290219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:32.353117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 901/1810] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:32.356543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:32.375576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7669f-c564-4c3f-8b44-6c89d86b4e2b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:32.401303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41e414-c997-4452-9afc-a4d7a125869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:32.402271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e414-c997-4452-9afc-a4d7a125869d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.409609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e414-c997-4452-9afc-a4d7a125869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.409945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e414-c997-4452-9afc-a4d7a125869d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.410609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41e414-c997-4452-9afc-a4d7a125869d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:32.412257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41e414-c997-4452-9afc-a4d7a125869d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:32.435664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 910/1811] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:32.440335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:32.457324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445adf6-9807-4580-a10c-798207ccdd01 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:32.467358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:32.469500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.469782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.470056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] PATH_INFO: `/v3/projects/b9f98b64520a4772a1b52fa7db3d21ca/users/a6664511f8c24d02a15092fd5affa6b1/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.495710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f98b64520a4772a1b52fa7db3d21ca, user_id=a6664511f8c24d02a15092fd5affa6b1, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:32.502632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:32.544864 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 902/1812] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:32 2022] PUT /identity/v3/projects/b9f98b64520a4772a1b52fa7db3d21ca/users/a6664511f8c24d02a15092fd5affa6b1/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:32.561017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.561534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.561804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.614589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] MFA Rules not processed for user `a6664511f8c24d02a15092fd5affa6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:32.616360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:32.719837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 911/1813] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:32.724845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:32.802627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4534b-9fcf-4e62-9cf0-4ecbfb1df547 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:32.808119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:32.809067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.809365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.809626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.810596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:32.812597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:32.871700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 903/1814] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:32 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:32.874821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:32.886316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176c518-5196-4b9d-8657-f762ee153a51 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:32.954738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:32.955307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:32.955590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:32.955861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:32.956536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:32.958608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:32.968314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:32.988736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 912/1815] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:32 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:32.992917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:33.009270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60eb7-86c5-4bca-a32a-0a4a58ba768a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:33.017090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:33.017737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.018001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.018271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.018953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:33.020794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:33.046525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 904/1816] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:33.047810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:33.058269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca515-df64-4e7c-a26d-4cc404495004 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:33.064403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:33.065240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.065523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.065799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] PATH_INFO: `/v3/projects/79990dcae1164b4cba3e7ded23517acc/users/8590bdfe348e4fc08aed39a1bf141ab2/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.082467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79990dcae1164b4cba3e7ded23517acc, user_id=8590bdfe348e4fc08aed39a1bf141ab2, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:33.085244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:33.100619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 913/1817] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:33 2022] PUT /identity/v3/projects/79990dcae1164b4cba3e7ded23517acc/users/8590bdfe348e4fc08aed39a1bf141ab2/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:33.105012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:33.115546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb9dec-01ce-4ab6-aba0-c7c2a0239beb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:33.168046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:33.191064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b286fd-612d-4725-8e8b-3a18b27adbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:33.191859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b286fd-612d-4725-8e8b-3a18b27adbaf admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.192287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b286fd-612d-4725-8e8b-3a18b27adbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.192664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b286fd-612d-4725-8e8b-3a18b27adbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.193441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b286fd-612d-4725-8e8b-3a18b27adbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:33.195216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b286fd-612d-4725-8e8b-3a18b27adbaf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:33.215809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 905/1818] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:33.217368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:33.234231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ad121-417d-4b1f-a970-f7dd35b71b27 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:33.239683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:33.240524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.240809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.245268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] PATH_INFO: `/v3/projects/79990dcae1164b4cba3e7ded23517acc/users/8590bdfe348e4fc08aed39a1bf141ab2/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.246970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79990dcae1164b4cba3e7ded23517acc, user_id=8590bdfe348e4fc08aed39a1bf141ab2, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:33.250452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:33.272069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 914/1819] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:33 2022] PUT /identity/v3/projects/79990dcae1164b4cba3e7ded23517acc/users/8590bdfe348e4fc08aed39a1bf141ab2/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:33.285205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.285655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.286024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.320994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] MFA Rules not processed for user `8590bdfe348e4fc08aed39a1bf141ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:33.322705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:33.428049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 906/1820] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:33.444588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:33.446749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:33.552516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924b22-9ec7-4df7-8580-152d53a3746f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:33.560173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:33.561334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.562079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.562079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.606550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:33.608551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:33.633429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 915/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:33.758200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:33.781150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc69f1-4748-42a3-beb5-0a52f9e22738 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:33.785453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74065c-9a78-48d4-97d5-a54e8568ca46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:33.786134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74065c-9a78-48d4-97d5-a54e8568ca46 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:33.786420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74065c-9a78-48d4-97d5-a54e8568ca46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:33.786691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74065c-9a78-48d4-97d5-a54e8568ca46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:33.875927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74065c-9a78-48d4-97d5-a54e8568ca46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:33.877423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74065c-9a78-48d4-97d5-a54e8568ca46 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:33.887879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 907/1822] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:34.984701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:34.986761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:35.008000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086c6ec-3961-4034-a3e8-86bf705a0edc service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:35.016005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:35.016577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:35.016845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:35.017325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:35.019506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:35.101768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:35.103223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7795c-ca07-401b-b1b3-89eaf449abe1 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:35.118807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 916/1823] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:37.239903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ec8b-06a0-4116-a1a4-3aef87155eaa None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:37.240523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ec8b-06a0-4116-a1a4-3aef87155eaa None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:37.240982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ec8b-06a0-4116-a1a4-3aef87155eaa None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:37.242539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 908/1824] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:16:37.274634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:37.275272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:37.275272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:37.295240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:37.296508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:37.347097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:37.367922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 917/1825] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:16:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:37.381337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ec8b-06a0-4116-a1a4-3aef87155eaa None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:37.383356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7f19ec8b-06a0-4116-a1a4-3aef87155eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:37.411643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ec8b-06a0-4116-a1a4-3aef87155eaa None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:37.417599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c1fc2-1ae0-4883-b2e6-ff1353e58f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:37.418324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c1fc2-1ae0-4883-b2e6-ff1353e58f0b service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:37.418673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c1fc2-1ae0-4883-b2e6-ff1353e58f0b service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:37.419016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c1fc2-1ae0-4883-b2e6-ff1353e58f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:37.429140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c1fc2-1ae0-4883-b2e6-ff1353e58f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:37.431154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c1fc2-1ae0-4883-b2e6-ff1353e58f0b service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:37.442580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 909/1826] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:42.271267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:42.393371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2fff2-c2bc-474d-bc3a-1ba7299fe38e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:42.400832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:42.420516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:42.421063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:42.421470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] PATH_INFO: `/v3/users/a6664511f8c24d02a15092fd5affa6b1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:42.427512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6664511f8c24d02a15092fd5affa6b1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:42.433452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:42.550610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6664511f8c24d02a15092fd5affa6b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:42.551153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6664511f8c24d02a15092fd5affa6b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:42.552027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6664511f8c24d02a15092fd5affa6b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:42.584719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6664511f8c24d02a15092fd5affa6b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:42.601716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6664511f8c24d02a15092fd5affa6b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:42.638226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 918/1827] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:42 2022] DELETE /identity/v3/users/a6664511f8c24d02a15092fd5affa6b1 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:42.645722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:42.646506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:42.646884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:42.729668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:42.731113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:42.813755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:42.839124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 910/1828] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:42.903707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:42.906632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:42.939124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a99c48-6765-4461-9afd-bdf81954d364 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:42.948125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:42.949284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:42.949561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:42.949957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] PATH_INFO: `/v3/users/98cbfae893e7462899c4f9e385c7f895` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:42.952797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98cbfae893e7462899c4f9e385c7f895)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:42.956043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:42.990746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:42.993817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.107761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98cbfae893e7462899c4f9e385c7f895'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.108830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98cbfae893e7462899c4f9e385c7f895'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.113426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98cbfae893e7462899c4f9e385c7f895'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.130972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:43.145341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98cbfae893e7462899c4f9e385c7f895'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.157157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98cbfae893e7462899c4f9e385c7f895'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.170659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bbd34-9f3f-4c2a-ba07-cae0e4221898 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:43.177389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:43.178623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:43.179108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:43.179553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 919/1829] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:42 2022] DELETE /identity/v3/users/98cbfae893e7462899c4f9e385c7f895 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:43.179695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:43.181014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:43.182387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.182695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.206602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:43.212300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e51888-4981-4997-9709-45ddab9e3993 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:43.257538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 911/1830] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:43.260475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:43.260987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:43.261488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:43.282962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30681c-0d58-4fc9-9acd-cc7b587905a3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:43.294140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:43.295708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:43.296325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:43.296926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] PATH_INFO: `/v3/users/1b1bb703ce4041e4a749b5df0c2d68e4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:43.299014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1bb703ce4041e4a749b5df0c2d68e4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:43.301976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:43.302803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:43.304312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.329056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 912/1831] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:43.400908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1bb703ce4041e4a749b5df0c2d68e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.402451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1bb703ce4041e4a749b5df0c2d68e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.403301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1bb703ce4041e4a749b5df0c2d68e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.410572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:43.413785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.433852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1bb703ce4041e4a749b5df0c2d68e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.464296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a810111a-570e-484a-b7c8-73dcb6b2bb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1bb703ce4041e4a749b5df0c2d68e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:43.528361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 920/1832] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:43 2022] DELETE /identity/v3/users/1b1bb703ce4041e4a749b5df0c2d68e4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:43.529845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:43.530138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:43.530407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:43.554935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76574c31-d754-442e-8ef2-d478f65f41af None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:43.571920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:43.573758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.670176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:43.674761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:43.678704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:43.678901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:43.679393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:43.682557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.694034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 921/1833] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:43.705567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:43.707366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:43.728957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:43.731182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:43.742006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 913/1834] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:43.767246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9ef6a-218a-402d-bfc9-6778f5a526a4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:43.777373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:43.780180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:43.784476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:43.785072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:43.803769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:43.806788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:43.824844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 922/1835] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:44.831900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:44.834327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:44.875119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d935d88-1536-4f4b-bfe3-9b12e6f44cce service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:44.892287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:44.893511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:44.894266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:44.894699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] PATH_INFO: `/v3/projects/b9f98b64520a4772a1b52fa7db3d21ca` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:44.898206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f98b64520a4772a1b52fa7db3d21ca)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:44.901196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:45.000489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:45.017587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f98b64520a4772a1b52fa7db3d21ca', 'request_id': 'req-1037bd8b-1602-43c7-aed4-e406f5db9e55'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.049354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f98b64520a4772a1b52fa7db3d21ca', 'request_id': 'req-1037bd8b-1602-43c7-aed4-e406f5db9e55'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.078447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f98b64520a4772a1b52fa7db3d21ca', 'request_id': 'req-1037bd8b-1602-43c7-aed4-e406f5db9e55'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.125341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] The token cache is being invalidate because project b9f98b64520a4772a1b52fa7db3d21ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:45.125942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.127021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.133395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 914/1836] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:44 2022] DELETE /identity/v3/projects/b9f98b64520a4772a1b52fa7db3d21ca => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:45.133995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:45.151548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507734c-1615-4597-9dc7-0ef0d0e33e85 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:45.228788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:45.230431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:45.295974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:45.296753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:45.297017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:45.297291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] PATH_INFO: `/v3/users/8590bdfe348e4fc08aed39a1bf141ab2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:45.300486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8590bdfe348e4fc08aed39a1bf141ab2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:45.304311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:45.397287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037bd8b-1602-43c7-aed4-e406f5db9e55 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:45.402943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:45.403591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:45.404168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:45.404940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] PATH_INFO: `/v3/projects/4e2523f00ead4f4995530e0f9b77356c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:45.406366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2523f00ead4f4995530e0f9b77356c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:45.408512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:45.449399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8590bdfe348e4fc08aed39a1bf141ab2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.450818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8590bdfe348e4fc08aed39a1bf141ab2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.452943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8590bdfe348e4fc08aed39a1bf141ab2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.507332 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8590bdfe348e4fc08aed39a1bf141ab2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.521740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2523f00ead4f4995530e0f9b77356c', 'request_id': 'req-581d9012-7d2c-40a0-abfb-59d664fae254'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.523809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8590bdfe348e4fc08aed39a1bf141ab2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.547582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2523f00ead4f4995530e0f9b77356c', 'request_id': 'req-581d9012-7d2c-40a0-abfb-59d664fae254'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.564268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 923/1837] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:45 2022] DELETE /identity/v3/users/8590bdfe348e4fc08aed39a1bf141ab2 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:45.567037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:45.580261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2523f00ead4f4995530e0f9b77356c', 'request_id': 'req-581d9012-7d2c-40a0-abfb-59d664fae254'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.590270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:45.634157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] The token cache is being invalidate because project 4e2523f00ead4f4995530e0f9b77356c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:45.634510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.636666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.651232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 915/1838] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:45 2022] DELETE /identity/v3/projects/4e2523f00ead4f4995530e0f9b77356c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:45.652213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:45.658297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:45.824683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae831584-caa0-4c05-8607-41a31da59af8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:45.842201 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:45.842873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:45.843158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:45.843455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] PATH_INFO: `/v3/projects/71d56cdb56f4479d95cec7b507d1e6a0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:45.849221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d56cdb56f4479d95cec7b507d1e6a0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:45.856065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:45.864097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:45.992324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d56cdb56f4479d95cec7b507d1e6a0', 'request_id': 'req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:45.994018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9012-7d2c-40a0-abfb-59d664fae254 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:46.008178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d56cdb56f4479d95cec7b507d1e6a0', 'request_id': 'req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.010024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:46.026993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d56cdb56f4479d95cec7b507d1e6a0', 'request_id': 'req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.050536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] The token cache is being invalidate because project 71d56cdb56f4479d95cec7b507d1e6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:46.051462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.052997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.054714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 924/1839] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:45 2022] DELETE /identity/v3/projects/71d56cdb56f4479d95cec7b507d1e6a0 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:46.059061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:46.061356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:46.141799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:46.142908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:46.143394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:46.143903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] PATH_INFO: `/v3/users/55ac62ee356941fb9c54c652ee2a5b34` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:46.152608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ac62ee356941fb9c54c652ee2a5b34)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:46.154768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2485a1-3dcc-4cdd-856d-6a911e54b5b2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:46.157547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:46.161261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:46.161916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:46.162191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:46.162463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] PATH_INFO: `/v3/users/6310e1a32f10439f9cdcbfbb57e82134` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:46.164495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6310e1a32f10439f9cdcbfbb57e82134)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:46.167523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:46.259049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ac62ee356941fb9c54c652ee2a5b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.286652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ac62ee356941fb9c54c652ee2a5b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.306338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6310e1a32f10439f9cdcbfbb57e82134'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.306985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6310e1a32f10439f9cdcbfbb57e82134'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.311362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6310e1a32f10439f9cdcbfbb57e82134'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.323652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ac62ee356941fb9c54c652ee2a5b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.325311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ac62ee356941fb9c54c652ee2a5b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.328349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ac62ee356941fb9c54c652ee2a5b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.336592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6310e1a32f10439f9cdcbfbb57e82134'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.352100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 916/1840] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:45 2022] DELETE /identity/v3/users/55ac62ee356941fb9c54c652ee2a5b34 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:46.357291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6310e1a32f10439f9cdcbfbb57e82134'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.375960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 925/1841] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:46 2022] DELETE /identity/v3/users/6310e1a32f10439f9cdcbfbb57e82134 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:46.508291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:46.510079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:46.615936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1c89-83af-4dc2-bf6e-29b031b1fe02 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:46.675918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:46.740382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:46.740382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:46.740382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:46.740382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] PATH_INFO: `/v3/projects/79990dcae1164b4cba3e7ded23517acc` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:46.743689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=79990dcae1164b4cba3e7ded23517acc)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:46.746515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:46.802346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:46.804301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:46.820831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7a0c9-302b-4ffd-8627-280b3890a01b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:46.836819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79990dcae1164b4cba3e7ded23517acc', 'request_id': 'req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.839348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:46.847296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79990dcae1164b4cba3e7ded23517acc', 'request_id': 'req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.857880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79990dcae1164b4cba3e7ded23517acc', 'request_id': 'req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.879915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] The token cache is being invalidate because project 79990dcae1164b4cba3e7ded23517acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:46.881296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.883002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e7f8d236-3443-4bf0-a80a-55e3c5add4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:46.885331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 917/1842] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:46 2022] DELETE /identity/v3/projects/79990dcae1164b4cba3e7ded23517acc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:46.891918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:46.892227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:46.892504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:46.914347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:46.915662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:46.923503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:46.924511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:46.924870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:46.925221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] PATH_INFO: `/v3/projects/c4b3df653cbb45ad9e02478c404d798b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:46.926750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b3df653cbb45ad9e02478c404d798b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:46.928941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.004829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 918/1843] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:47.004829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.008345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b3df653cbb45ad9e02478c404d798b', 'request_id': 'req-7748f0fc-9323-40c2-89b2-775695982bef'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.012985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.020378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b3df653cbb45ad9e02478c404d798b', 'request_id': 'req-7748f0fc-9323-40c2-89b2-775695982bef'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.037255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b3df653cbb45ad9e02478c404d798b', 'request_id': 'req-7748f0fc-9323-40c2-89b2-775695982bef'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.040063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30b75-ffc2-46c0-b9a5-bbec96b979d2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.053436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] The token cache is being invalidate because project c4b3df653cbb45ad9e02478c404d798b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:47.053738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.054496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.056278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 926/1844] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:46 2022] DELETE /identity/v3/projects/c4b3df653cbb45ad9e02478c404d798b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:47.057076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.059068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.116684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.117360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.117892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.118184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] PATH_INFO: `/v3/users/8684a435be704fb79600b32813b276b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.119490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8684a435be704fb79600b32813b276b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.121791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.128378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:47.140581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748f0fc-9323-40c2-89b2-775695982bef admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.148809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.149510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.149845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.150047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] PATH_INFO: `/v3/projects/57aea120f26c4ec0803a993cded94271` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.151163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=57aea120f26c4ec0803a993cded94271)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.153114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.176942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8684a435be704fb79600b32813b276b3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.193905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8684a435be704fb79600b32813b276b3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.203747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8684a435be704fb79600b32813b276b3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.204003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:47.204468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8684a435be704fb79600b32813b276b3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.205107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-befd99ee-616b-4489-a1a9-3c5b219972b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8684a435be704fb79600b32813b276b3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.209635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57aea120f26c4ec0803a993cded94271', 'request_id': 'req-32d113ab-e490-4f2b-9b47-9e44ab724bee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.218791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 919/1845] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:46 2022] DELETE /identity/v3/users/8684a435be704fb79600b32813b276b3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:47.220067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8036a-5bd1-490c-9b17-977dd930a8c3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.220510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8036a-5bd1-490c-9b17-977dd930a8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.220805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8036a-5bd1-490c-9b17-977dd930a8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.221865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57aea120f26c4ec0803a993cded94271', 'request_id': 'req-32d113ab-e490-4f2b-9b47-9e44ab724bee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.230948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57aea120f26c4ec0803a993cded94271', 'request_id': 'req-32d113ab-e490-4f2b-9b47-9e44ab724bee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.241544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-eac8036a-5bd1-490c-9b17-977dd930a8c3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:47.242757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eac8036a-5bd1-490c-9b17-977dd930a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.243221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] The token cache is being invalidate because project 57aea120f26c4ec0803a993cded94271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:47.243509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.244170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.245727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 927/1846] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:47 2022] DELETE /identity/v3/projects/57aea120f26c4ec0803a993cded94271 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:47.246567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.248688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.315484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 920/1847] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:47.317103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2b675-d6ca-42d9-bd40-f0845b75ad17 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.317380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2b675-d6ca-42d9-bd40-f0845b75ad17 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.317655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2b675-d6ca-42d9-bd40-f0845b75ad17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.325496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d113ab-e490-4f2b-9b47-9e44ab724bee admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.332783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.333430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.333700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.333976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] PATH_INFO: `/v3/users/a0be7773ede24ed18f4191fd18651399` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.335150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0be7773ede24ed18f4191fd18651399)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.337111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.343670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e2d2b675-d6ca-42d9-bd40-f0845b75ad17 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:47.344959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2d2b675-d6ca-42d9-bd40-f0845b75ad17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.353877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 921/1848] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:47.354943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.355227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.355497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.384413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:47.385655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.396950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 922/1849] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:47.397798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.418791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efa24d-8a56-4e32-b5fc-a0465b434159 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.428182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.428834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.429109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.429372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.430075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:47.430375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.432169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.440516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0be7773ede24ed18f4191fd18651399'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.442038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0be7773ede24ed18f4191fd18651399'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.442765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0be7773ede24ed18f4191fd18651399'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.447762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 923/1850] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:47.448426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.450847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.463741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0be7773ede24ed18f4191fd18651399'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.475611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e65cc428-bee4-4181-9977-170381ecddee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0be7773ede24ed18f4191fd18651399'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:47.495460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 928/1851] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:47 2022] DELETE /identity/v3/users/a0be7773ede24ed18f4191fd18651399 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:47.500408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.500750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.500993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.527762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:47.529202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.546347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e567bb9-b00b-482d-9035-980ae5d6d45d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.566316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:47.608377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:47.643024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 929/1852] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:47.643817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.646212 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.662809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073b164-40a0-4bed-9c0c-2b0fb79d6d0f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.663701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.664425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.664674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.664954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.666881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:47.668222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.668621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.668902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.669173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.670438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.672503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.681989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.683435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.694546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 924/1853] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:47.697190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.707627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23b190-4eac-46c4-992c-433288aa0108 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.708224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 930/1854] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:47.713008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.801651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.802412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.802791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.803284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.804138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:47.804687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.807274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.829312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76370a5d-a345-4263-a5aa-26d0ca422da0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.836023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.836734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.837466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.837778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.838713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.840945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.846017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 925/1855] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:47.848099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.869224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174e30d-9137-4bb9-8baa-17c039e4a5d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.878535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 931/1856] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:47.879864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.885169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.886509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.887417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.889088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.890790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.893540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb44517-b297-4fb9-8317-f143997fc955 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.894997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.901047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:47.901927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:47.902222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:47.902504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:47.903420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:47.906101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:47.920825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 932/1857] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:47.926729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:47.943238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe63e9-b386-42cb-95d4-a96fb634cbfd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:47.948068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 926/1858] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:47.953307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.053307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.054380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.054380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.054603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.055297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.057097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.057211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b95c-299a-4ac8-bdc1-196f8feee6d9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.062670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.063419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.064245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.064245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.065016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.067581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.073165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 933/1859] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:48.075288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.087214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab679cb-c750-48c5-8ab9-98930cdd0378 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.093944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084084ff-7d4f-4b52-909a-b054a61c42e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.094701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084084ff-7d4f-4b52-909a-b054a61c42e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.094995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084084ff-7d4f-4b52-909a-b054a61c42e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.095265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084084ff-7d4f-4b52-909a-b054a61c42e2 admin admin] PATH_INFO: `/v3/projects/022e8929e87f4920ac4871281b1223fd/users/1f9695d1a8924f05a4010cc8d6690a5d/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.105767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 927/1860] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:47 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:48.108486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.113960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084084ff-7d4f-4b52-909a-b054a61c42e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=022e8929e87f4920ac4871281b1223fd, user_id=1f9695d1a8924f05a4010cc8d6690a5d, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.116755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084084ff-7d4f-4b52-909a-b054a61c42e2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.121877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59744b3-cce9-4873-9c53-d187a1cdc85e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.131896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.133345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.133703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.134051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.136750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.140334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.144671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 934/1861] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:48 2022] PUT /identity/v3/projects/022e8929e87f4920ac4871281b1223fd/users/1f9695d1a8924f05a4010cc8d6690a5d/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:48.155738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.159001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 928/1862] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:48.159240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.159665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.163151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.175908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2ca4-957a-40ba-8662-85bafdc5d3ff admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.194196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:48.207555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] MFA Rules not processed for user `1f9695d1a8924f05a4010cc8d6690a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:48.209897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:48.226561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:48.287996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.289255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.290307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.290429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.292594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.312642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.335414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 929/1863] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:48.345347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.368995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4354f6-8f1f-4662-814b-79fcccf3c5d7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.396060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.397277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.397663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.398158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] PATH_INFO: `/v3/projects/c546091e7fdb4b92ac933f3a2d8fa321/users/514a86d1fcd34392ad0557722f0839df/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.423824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c546091e7fdb4b92ac933f3a2d8fa321, user_id=514a86d1fcd34392ad0557722f0839df, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.427451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.433522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 935/1864] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:48.435032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.437694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:48.451866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 930/1865] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:48 2022] PUT /identity/v3/projects/c546091e7fdb4b92ac933f3a2d8fa321/users/514a86d1fcd34392ad0557722f0839df/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:48.457260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.477211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965b825-10eb-4566-8ff1-480542fe74fa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.550243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa639c6-132d-4742-b0a3-63256d78370f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.555769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165abd8-9c95-45bd-8a9f-db31293f1b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.556639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165abd8-9c95-45bd-8a9f-db31293f1b23 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.556793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165abd8-9c95-45bd-8a9f-db31293f1b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.557156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165abd8-9c95-45bd-8a9f-db31293f1b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.557670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165abd8-9c95-45bd-8a9f-db31293f1b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.558546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.559250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.560020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.560916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165abd8-9c95-45bd-8a9f-db31293f1b23 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.561045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] PATH_INFO: `/v3/projects/78173d6b8dd44eb29b41afda05abf06b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.562271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78173d6b8dd44eb29b41afda05abf06b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.564281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.601336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 931/1866] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:48 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:48.602758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.603050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.603490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.629575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78173d6b8dd44eb29b41afda05abf06b', 'request_id': 'req-ceedc8f3-a169-4690-90af-d9439bf0e6a0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.636299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] MFA Rules not processed for user `514a86d1fcd34392ad0557722f0839df`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:48.637609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:48.640575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78173d6b8dd44eb29b41afda05abf06b', 'request_id': 'req-ceedc8f3-a169-4690-90af-d9439bf0e6a0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.648751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78173d6b8dd44eb29b41afda05abf06b', 'request_id': 'req-ceedc8f3-a169-4690-90af-d9439bf0e6a0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.693264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] The token cache is being invalidate because project 78173d6b8dd44eb29b41afda05abf06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:48.694150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.695618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.704939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 936/1867] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:48 2022] DELETE /identity/v3/projects/78173d6b8dd44eb29b41afda05abf06b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:48.706703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.716263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:48.750953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:48.768246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 932/1868] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:48.769267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.773865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:48.825761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:48.856723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedc8f3-a169-4690-90af-d9439bf0e6a0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.862674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.863562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.863846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.864115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] PATH_INFO: `/v3/projects/bcd0b12fb80a401690dd728c318968a0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.865335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd0b12fb80a401690dd728c318968a0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.865731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911f3ab-2c1c-41a3-b6a7-9d36e06dcc60 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:48.867596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.871688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83242f-6609-4867-8073-41d3fc01fcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:48.872337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83242f-6609-4867-8073-41d3fc01fcaf admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.872834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83242f-6609-4867-8073-41d3fc01fcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.873485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83242f-6609-4867-8073-41d3fc01fcaf admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.874059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83242f-6609-4867-8073-41d3fc01fcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:48.876037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83242f-6609-4867-8073-41d3fc01fcaf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:48.909555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 933/1869] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:48 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:48.909555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:48.909555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:48.909555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:48.929854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:48.930965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:48.944281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd0b12fb80a401690dd728c318968a0', 'request_id': 'req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.955911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd0b12fb80a401690dd728c318968a0', 'request_id': 'req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.968615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd0b12fb80a401690dd728c318968a0', 'request_id': 'req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.981667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] The token cache is being invalidate because project bcd0b12fb80a401690dd728c318968a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:48.981957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.983287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:48.984837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 937/1870] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:48 2022] DELETE /identity/v3/projects/bcd0b12fb80a401690dd728c318968a0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:48.985581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:48.990073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.015217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 934/1871] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:49.016675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.018118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.034280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2fbde-a463-4fe9-851d-1461c597cc84 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.040050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.040704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.040984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.041260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.041903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.043515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.071669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35f02-cc0a-4fb4-a51e-f644a9562a65 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.072099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 935/1872] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:49.073598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.077305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.080098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ec2cd-1a0b-4781-a8d2-38efb1216b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.080927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec2cd-1a0b-4781-a8d2-38efb1216b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.081271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec2cd-1a0b-4781-a8d2-38efb1216b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.081609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec2cd-1a0b-4781-a8d2-38efb1216b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.082303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ec2cd-1a0b-4781-a8d2-38efb1216b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.084328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ec2cd-1a0b-4781-a8d2-38efb1216b92 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.102277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4735d-318b-43e7-8d9c-0af4e38bf032 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.110840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.111543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.111955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.112389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] PATH_INFO: `/v3/users/1803033809f4487bbb488636bca651c2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.114639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1803033809f4487bbb488636bca651c2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.118061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.119071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 938/1873] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:48 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:49.120137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.120536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.120944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.148794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:49.150229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.182572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1803033809f4487bbb488636bca651c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:49.197819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1803033809f4487bbb488636bca651c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:49.201982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:49.211226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:49.213618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1803033809f4487bbb488636bca651c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:49.214340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1803033809f4487bbb488636bca651c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:49.215391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1803033809f4487bbb488636bca651c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:49.227757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 939/1874] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:49.228665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.230034 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.241856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 936/1875] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:49 2022] DELETE /identity/v3/users/1803033809f4487bbb488636bca651c2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:49.241856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.242957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276406-f9f7-4d5c-b064-8de1dc454ccb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.252162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.325345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.327023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.327637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.328007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] PATH_INFO: `/v3/projects/02f607a953e94bc6814474ac8bfd25f1/users/cc8e10af01e84a5fa3bcf71a1e1815e7/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.346647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f607a953e94bc6814474ac8bfd25f1, user_id=cc8e10af01e84a5fa3bcf71a1e1815e7, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.349551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.365358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 940/1876] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:49 2022] PUT /identity/v3/projects/02f607a953e94bc6814474ac8bfd25f1/users/cc8e10af01e84a5fa3bcf71a1e1815e7/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:49.368473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.369944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceca747-9458-4fa3-89c3-f434e791a552 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.464490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.465443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.466091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.466974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.468001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.470599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.491496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d93a7-a8a6-4207-ac6f-a18616b6a4a5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.498873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.499858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.500291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.500663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.501052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 937/1877] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:49 2022] POST /identity/v3/users => generated 300 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:49.501563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:49.501898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.502252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.503921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.513516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dca0b-9717-4fe0-b0a2-410624f5c73b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.522945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 941/1878] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:49.523159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.523390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.523503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.523503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.523503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.523503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.524321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.535110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b41d7-d463-4c0b-8fa5-9284f0bfc655 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.540249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 938/1879] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:49.540249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.545580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.546573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.547021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.547472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.548604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.552056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.563517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c01d8f-326f-4ee8-9de9-48ab31f4530d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.573380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 942/1880] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:49.574687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.578521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c87bbc-89b7-453d-9dc8-5852a9652b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.579352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c87bbc-89b7-453d-9dc8-5852a9652b23 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.585697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c87bbc-89b7-453d-9dc8-5852a9652b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.586005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c87bbc-89b7-453d-9dc8-5852a9652b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.586645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c87bbc-89b7-453d-9dc8-5852a9652b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.588631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c87bbc-89b7-453d-9dc8-5852a9652b23 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.589679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceab49-5ae4-4c21-a49d-c6ab80349cf1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.595045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.596238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.596238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.596453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] PATH_INFO: `/v3/projects/02f607a953e94bc6814474ac8bfd25f1/users/cc8e10af01e84a5fa3bcf71a1e1815e7/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.598311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f607a953e94bc6814474ac8bfd25f1, user_id=cc8e10af01e84a5fa3bcf71a1e1815e7, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.601062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.618836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 943/1881] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:49 2022] PUT /identity/v3/projects/02f607a953e94bc6814474ac8bfd25f1/users/cc8e10af01e84a5fa3bcf71a1e1815e7/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:49.619857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.638537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 939/1882] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:49 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:49.638537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.638537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.638537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.672715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] MFA Rules not processed for user `cc8e10af01e84a5fa3bcf71a1e1815e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:49.673978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.685316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:49.705351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23375a-c28c-4941-bade-a507452e3510 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.725999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:49.798475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 940/1883] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:49.799426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.799777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.800948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.800948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.801229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] PATH_INFO: `/v3/projects/a78cdef84e444f8d9e4991a8281f4c85/users/ff14869177b348e1816aa1dcdebfb5db/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.813989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e1961-dee5-465f-80c6-74c289e79ed8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.820234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.821089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.821539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.821808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.822316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78cdef84e444f8d9e4991a8281f4c85, user_id=ff14869177b348e1816aa1dcdebfb5db, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.822440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.825399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.827717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.846085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 944/1884] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:49 2022] PUT /identity/v3/projects/a78cdef84e444f8d9e4991a8281f4c85/users/ff14869177b348e1816aa1dcdebfb5db/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:49.846996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.848610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 941/1885] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:49 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:49.848747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.849773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:49.853374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:49.915639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5ed25-22a2-4c6e-b0f3-9baaaf77080b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.928057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814be72-7df0-43cf-a5a8-aac729a83f5e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:49.933712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.934396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.934672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.934942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] PATH_INFO: `/v3/projects/19838c946271462a9d25b3356beb9575` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:49.936234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] RBAC: Authorizing `identity:delete_project(project_id=19838c946271462a9d25b3356beb9575)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:49.938271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:49.983259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:49.984044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:49.984401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:49.984702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.012567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19838c946271462a9d25b3356beb9575', 'request_id': 'req-ff81558c-aac5-4b34-a58e-eea126a8809a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:50.028114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19838c946271462a9d25b3356beb9575', 'request_id': 'req-ff81558c-aac5-4b34-a58e-eea126a8809a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:50.041342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19838c946271462a9d25b3356beb9575', 'request_id': 'req-ff81558c-aac5-4b34-a58e-eea126a8809a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:50.058005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] The token cache is being invalidate because project 19838c946271462a9d25b3356beb9575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:50.058306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:50.059134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:50.062367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 945/1886] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:49 2022] DELETE /identity/v3/projects/19838c946271462a9d25b3356beb9575 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:50.063500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.064128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.079484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:50.114197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.115739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.127924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 942/1887] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:50.143460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.172321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2a2e7-acaf-4906-a8b1-ed51185aa77e service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.175064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81558c-aac5-4b34-a58e-eea126a8809a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.180434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837725-2371-4b7a-b3bf-010f6fce4d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.181272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837725-2371-4b7a-b3bf-010f6fce4d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.181691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837725-2371-4b7a-b3bf-010f6fce4d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.182130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837725-2371-4b7a-b3bf-010f6fce4d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.182960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07837725-2371-4b7a-b3bf-010f6fce4d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.183341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.184127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.184577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.184807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07837725-2371-4b7a-b3bf-010f6fce4d94 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.186883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.187878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.193498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.199405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 946/1888] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:50.202134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.202134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.202369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.218868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 943/1889] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:50.219924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.244830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:50.248387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30808ccd-1189-441d-b619-72291c8c8fb4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.250541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.261426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8676708-64ee-45b9-8533-90a9f2a9818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.262124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8676708-64ee-45b9-8533-90a9f2a9818a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.262400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8676708-64ee-45b9-8533-90a9f2a9818a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.262684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8676708-64ee-45b9-8533-90a9f2a9818a admin admin] PATH_INFO: `/v3/projects/a78cdef84e444f8d9e4991a8281f4c85/users/ff14869177b348e1816aa1dcdebfb5db/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.267306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8676708-64ee-45b9-8533-90a9f2a9818a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78cdef84e444f8d9e4991a8281f4c85, user_id=ff14869177b348e1816aa1dcdebfb5db, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.270388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 947/1890] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:50.277585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.281274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.283650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8676708-64ee-45b9-8533-90a9f2a9818a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.298139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:50.315252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a623d8-9b06-45b3-9375-0deadc852be0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.317993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 944/1891] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:50 2022] PUT /identity/v3/projects/a78cdef84e444f8d9e4991a8281f4c85/users/ff14869177b348e1816aa1dcdebfb5db/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:50.321556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.321999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.322478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.410295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:50.411722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.517934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84960d-63ce-4491-8031-8362dfb176e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.518510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84960d-63ce-4491-8031-8362dfb176e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.518787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84960d-63ce-4491-8031-8362dfb176e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.519059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84960d-63ce-4491-8031-8362dfb176e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.519719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84960d-63ce-4491-8031-8362dfb176e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.521502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84960d-63ce-4491-8031-8362dfb176e9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.554857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 948/1892] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:50.554857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.557194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.558510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.595742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 945/1893] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:50.597932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.602679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.642836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] MFA Rules not processed for user `ff14869177b348e1816aa1dcdebfb5db`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:50.645525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.717792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4820f-8343-4000-845f-f28c02ce341e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.729482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.730398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.730759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.731913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.737652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.849360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 949/1894] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:50.850119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.879765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdea25-257a-4cb8-ae77-0a361bb07104 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.895523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.897500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.897500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.897500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] PATH_INFO: `/v3/projects/6be6791ac9da47afb8a5fb18392b31f2/users/37fa87c3377a4cafb1b24ce2ba37a348/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.907966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.910076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.934516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 946/1895] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:50.936901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.938775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be6791ac9da47afb8a5fb18392b31f2, user_id=37fa87c3377a4cafb1b24ce2ba37a348, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:50.944005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:50.953337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:50.959794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22acbd-b44e-42f0-b7be-044e0250ee59 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:50.973761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 950/1896] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:50 2022] PUT /identity/v3/projects/6be6791ac9da47afb8a5fb18392b31f2/users/37fa87c3377a4cafb1b24ce2ba37a348/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:50.975385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:50.983814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:50.986805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:50.987589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:50.988847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:50.989354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:50.990773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:50.992117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.006774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d489e3-ea2a-4adb-9472-c0f48973e330 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.061899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 947/1897] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:51.068490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.069036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.069562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.119818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] MFA Rules not processed for user `37fa87c3377a4cafb1b24ce2ba37a348`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:51.122243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:51.176733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:51.197025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a582b-c99e-420a-b3b9-43f8a97b92f0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.206619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eae51-8990-48bf-92b0-2b0b96440b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.207515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eae51-8990-48bf-92b0-2b0b96440b90 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.208063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eae51-8990-48bf-92b0-2b0b96440b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.208495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eae51-8990-48bf-92b0-2b0b96440b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.259444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 948/1898] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:51.259444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:51.416759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eae51-8990-48bf-92b0-2b0b96440b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.420019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eae51-8990-48bf-92b0-2b0b96440b90 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.426944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcd352-7c31-4ac6-b4af-47e1b9be47e0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.444091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164c243-37ea-45d0-b5bb-813b69c51bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.444979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164c243-37ea-45d0-b5bb-813b69c51bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.448655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164c243-37ea-45d0-b5bb-813b69c51bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.448655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164c243-37ea-45d0-b5bb-813b69c51bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.455881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164c243-37ea-45d0-b5bb-813b69c51bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.458528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164c243-37ea-45d0-b5bb-813b69c51bdd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.461319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 951/1899] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:51.469371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73f4e-c32b-44dc-9ecf-f1ced775f9a7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.473304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73f4e-c32b-44dc-9ecf-f1ced775f9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.473811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73f4e-c32b-44dc-9ecf-f1ced775f9a7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.476181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 952/1900] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:16:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:16:51.477419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73f4e-c32b-44dc-9ecf-f1ced775f9a7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:51.508662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73f4e-c32b-44dc-9ecf-f1ced775f9a7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.521238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.522495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.522956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.525931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.526973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.532814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 949/1901] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:51 2022] POST /identity/v3/projects => generated 385 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:51.534379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.534700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.534815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.536032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.594856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:51.599871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:51.611789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 953/1902] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:51 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:51.611789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:51.824059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 950/1903] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:16:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:51.828125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:51.840451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089a8d-cbc0-49f7-b7a3-776f8573e19b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.859272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.860330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.860330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.860922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.861510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.866421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.870617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14955de5-47dc-4aa3-9084-8f5e449fff41 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.878264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.879221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.879648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.880083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.880935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.883980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.918423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 954/1904] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:51 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:51.919615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:51.928823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:51.948276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3611-2b22-4bc0-a5ef-b383cd0b095e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.954222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 951/1905] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:51 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:51.956995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:51.957890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.958688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.959071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.959441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] PATH_INFO: `/v3/users/1f9695d1a8924f05a4010cc8d6690a5d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.977205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9695d1a8924f05a4010cc8d6690a5d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.984542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.988066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a38dd2-4bd4-4c32-93ca-c03da01abbb5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:51.993620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:51.994253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:51.994527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:51.994797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:51.995441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:51.997089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:51.997998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:52.034674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 952/1906] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:52.035708 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.071445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6887e2-2609-46af-a6d5-13e4dca77a53 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.086309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.087461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.088175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.089737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.090432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:52.093415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:52.126760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 953/1907] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:52.128333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.141412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9695d1a8924f05a4010cc8d6690a5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:52.141412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9695d1a8924f05a4010cc8d6690a5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:52.141412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9695d1a8924f05a4010cc8d6690a5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:52.156865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b0b2-71b3-4f1a-9a7b-b1c465a12334 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.164314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.165273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.165554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.165828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.166657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:52.168786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:52.172496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9695d1a8924f05a4010cc8d6690a5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:52.173219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:52.195343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9695d1a8924f05a4010cc8d6690a5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:52.220449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 954/1908] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:52.221857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.226466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:52.234788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 955/1909] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:51 2022] DELETE /identity/v3/users/1f9695d1a8924f05a4010cc8d6690a5d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:52.235963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.239018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:52.268309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d8b4-569c-424f-8bef-abfb0e6a8d8f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.402272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babccbb-8e07-4cd5-9e3a-4c46ec95658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.403750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babccbb-8e07-4cd5-9e3a-4c46ec95658f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.404179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babccbb-8e07-4cd5-9e3a-4c46ec95658f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.406357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babccbb-8e07-4cd5-9e3a-4c46ec95658f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.406357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babccbb-8e07-4cd5-9e3a-4c46ec95658f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:52.408100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babccbb-8e07-4cd5-9e3a-4c46ec95658f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:52.436927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f90542-034a-400e-80ca-b4819703b0de admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.446077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 955/1910] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:52.447586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.447789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.449330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.470489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.471178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.471456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.471771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] PATH_INFO: `/v3/projects/22faf57641344f809c2aaacb1afeb193/users/6a1edb063abb419b8618e763558db7a9/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.529784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:52.529784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:52.530058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22faf57641344f809c2aaacb1afeb193, user_id=6a1edb063abb419b8618e763558db7a9, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:52.530795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:52.551009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 956/1911] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:52.556800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.558896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:52.568934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 956/1912] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:52 2022] PUT /identity/v3/projects/22faf57641344f809c2aaacb1afeb193/users/6a1edb063abb419b8618e763558db7a9/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:52.586892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.770789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff3ee8-def1-4fe6-bc2d-e145cf22f326 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.787182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.787850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.788125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.788471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] PATH_INFO: `/v3/projects/9f299c8eb7a549c0841a1ca425995a19/users/09adf9fc45144607ba068a83e2c80186/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.792474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1350c04-dc7a-414d-b8a8-ec6a52c74b98 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.816255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:52.821263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f299c8eb7a549c0841a1ca425995a19, user_id=09adf9fc45144607ba068a83e2c80186, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:52.828641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:52.865514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 957/1913] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:52 2022] PUT /identity/v3/projects/9f299c8eb7a549c0841a1ca425995a19/users/09adf9fc45144607ba068a83e2c80186/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:52.867500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:52.880480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:52.960903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.961809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.962467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.963058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:52.968915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:52.991301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35686115-0fdb-47a1-bcc3-235f7d533a3b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:52.995969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c846b6d-5e06-4d9e-be1e-c143e9298eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:52.996936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c846b6d-5e06-4d9e-be1e-c143e9298eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:52.997417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c846b6d-5e06-4d9e-be1e-c143e9298eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:52.997579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:52.998641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c846b6d-5e06-4d9e-be1e-c143e9298eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.001608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b77d6c-71b9-4efd-bb11-2d2839df6a14 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.020634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 957/1914] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:53.022464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.022969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.023488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.057994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:53.070920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] MFA Rules not processed for user `6a1edb063abb419b8618e763558db7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:53.072827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.094784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c846b6d-5e06-4d9e-be1e-c143e9298eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.096253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c846b6d-5e06-4d9e-be1e-c143e9298eb3 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.107907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 958/1915] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:53.109551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.109799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.110071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.156319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] MFA Rules not processed for user `09adf9fc45144607ba068a83e2c80186`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:53.157637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.179958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 958/1916] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:53.180921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.196607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.223563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393af75-6ebf-4893-bf64-b30400fddd2d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.230802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.232849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.233343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.234488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] PATH_INFO: `/v3/users/514a86d1fcd34392ad0557722f0839df` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.236698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=514a86d1fcd34392ad0557722f0839df)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.238884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.248291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 959/1917] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:53.248291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.262248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92cac-c307-4daf-be12-8a7710f8b7e6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.269866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.270500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.270924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.271204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.271875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.274140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.326776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 960/1918] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:53.331085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.343275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514a86d1fcd34392ad0557722f0839df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.343810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:53.379332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514a86d1fcd34392ad0557722f0839df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.390955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514a86d1fcd34392ad0557722f0839df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.393245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514a86d1fcd34392ad0557722f0839df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.394531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514a86d1fcd34392ad0557722f0839df'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.425723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 959/1919] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:53 2022] DELETE /identity/v3/users/514a86d1fcd34392ad0557722f0839df => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:53.426805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.429331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.465600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7045-f2e5-497d-a3bf-d516a01478bb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.540813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47aea3-8227-49e9-b0f7-227025d48373 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.546030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a435d7-ea94-4b79-8239-b440e8c2c424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.546628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a435d7-ea94-4b79-8239-b440e8c2c424 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.546907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a435d7-ea94-4b79-8239-b440e8c2c424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.547178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a435d7-ea94-4b79-8239-b440e8c2c424 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.547807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a435d7-ea94-4b79-8239-b440e8c2c424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.550002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a435d7-ea94-4b79-8239-b440e8c2c424 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.573398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.574460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.574888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.575337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.577065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.582547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 960/1920] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:53.582547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.582547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.582547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.585676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.598481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:53.614545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:53.615759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.619676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 961/1921] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/projects => generated 379 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:53.619676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.619676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.680061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 961/1922] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:53.681103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.686535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e83bd0-7349-4625-96d6-c303ad11eff8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.691388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.692075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.692409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.692670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] PATH_INFO: `/v3/projects/022e8929e87f4920ac4871281b1223fd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.693749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d258f-2010-44a7-84ca-26edec32df2e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.693864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=022e8929e87f4920ac4871281b1223fd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.695787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.700821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.702952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.702952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.704825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.704825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.706724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.716703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 962/1923] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:53.717742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.720005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.733060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b862f97-6d18-4652-b6e1-c1ced3223732 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.738485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.739096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.739378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.739655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.740311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.742251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.755037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:53.759294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022e8929e87f4920ac4871281b1223fd', 'request_id': 'req-a92042b1-956e-48c8-8504-a1067a4fda90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.768522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 963/1924] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:53 2022] POST /identity/v3/users => generated 296 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:53.769630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.771291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.774197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022e8929e87f4920ac4871281b1223fd', 'request_id': 'req-a92042b1-956e-48c8-8504-a1067a4fda90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.781778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022e8929e87f4920ac4871281b1223fd', 'request_id': 'req-a92042b1-956e-48c8-8504-a1067a4fda90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.792541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] The token cache is being invalidate because project 022e8929e87f4920ac4871281b1223fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:53.792833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.793686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:53.795267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 962/1925] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:53 2022] DELETE /identity/v3/projects/022e8929e87f4920ac4871281b1223fd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:53.796266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.797736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.834949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857d30-7219-4884-867c-881eefd2f075 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.866532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92042b1-956e-48c8-8504-a1067a4fda90 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.872539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.873529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.873915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.874411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] PATH_INFO: `/v3/projects/3ee68f858a84444f8b657e95f4702566/users/3fdeb3d570f04669b98c8adbd49c4567/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.893229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee68f858a84444f8b657e95f4702566, user_id=3fdeb3d570f04669b98c8adbd49c4567, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.894936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:53.895555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:53.895835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:53.896109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:53.897902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.898274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.913030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:53.913441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 963/1926] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:53 2022] PUT /identity/v3/projects/3ee68f858a84444f8b657e95f4702566/users/3fdeb3d570f04669b98c8adbd49c4567/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:53.914693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:53.917036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.918647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.925455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 964/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:53.926357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:53.927841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:53.940507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a39c61-8bd7-49ed-8b0f-22c173edd32a service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:53.988152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:54.004085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3239a9-175c-4638-9531-6f559397fa63 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.009797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.010426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.010759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.011190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.011890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.013694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.023699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.025603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.025603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.025603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] PATH_INFO: `/v3/users/cc8e10af01e84a5fa3bcf71a1e1815e7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.025832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 964/1928] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:54.028295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8e10af01e84a5fa3bcf71a1e1815e7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.029270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.029797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.041447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ee84c-8472-47b8-b480-adec1b3f76a5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.047679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.048363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.048699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.048856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.049811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.051385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.068242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 965/1929] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:54.069230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.091649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fdd58-a5be-4684-bd18-4703ec1730fd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.097585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8e10af01e84a5fa3bcf71a1e1815e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.100266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.101938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.102640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.103523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] PATH_INFO: `/v3/projects/c170d396473c44f48dea5e372c055ba0/users/21be934b320b4d64868e677d0ff8d552/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.114721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8e10af01e84a5fa3bcf71a1e1815e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.127485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8e10af01e84a5fa3bcf71a1e1815e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.128092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c170d396473c44f48dea5e372c055ba0, user_id=21be934b320b4d64868e677d0ff8d552, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.128406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8e10af01e84a5fa3bcf71a1e1815e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.129124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8e10af01e84a5fa3bcf71a1e1815e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.130833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.144287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 965/1930] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:53 2022] DELETE /identity/v3/users/cc8e10af01e84a5fa3bcf71a1e1815e7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:54.145085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.146527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.155188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 966/1931] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:54 2022] PUT /identity/v3/projects/c170d396473c44f48dea5e372c055ba0/users/21be934b320b4d64868e677d0ff8d552/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:54.156336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.158899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.186155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:54.244519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695fc37-c9b5-4963-bbc9-cc947d144ac9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.260485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.260764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa884cc-0d02-4dc4-8152-08d704615938 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.261925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.262372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.262481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.263162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.265003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.274721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5936d2-a145-479d-99d6-54150252987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.276645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5936d2-a145-479d-99d6-54150252987e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.277311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5936d2-a145-479d-99d6-54150252987e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.277311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5936d2-a145-479d-99d6-54150252987e admin admin] PATH_INFO: `/v3/projects/3ee68f858a84444f8b657e95f4702566/users/3fdeb3d570f04669b98c8adbd49c4567/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.283382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5936d2-a145-479d-99d6-54150252987e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee68f858a84444f8b657e95f4702566, user_id=3fdeb3d570f04669b98c8adbd49c4567, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.286392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5936d2-a145-479d-99d6-54150252987e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.293511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 967/1932] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:54.298250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.316440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba20e2-bdbf-4b93-b5a3-de317e562235 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.317708 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 966/1933] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:54 2022] PUT /identity/v3/projects/3ee68f858a84444f8b657e95f4702566/users/3fdeb3d570f04669b98c8adbd49c4567/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:54.330428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd337c-930d-4a51-8afc-27739a8fca27 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.330776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd337c-930d-4a51-8afc-27739a8fca27 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.331088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd337c-930d-4a51-8afc-27739a8fca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.374980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3fdd337c-930d-4a51-8afc-27739a8fca27 None None] MFA Rules not processed for user `3fdeb3d570f04669b98c8adbd49c4567`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:54.376891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3fdd337c-930d-4a51-8afc-27739a8fca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.444063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.456222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.457668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.458135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:54.458926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] PATH_INFO: `/v3/projects/c170d396473c44f48dea5e372c055ba0/users/21be934b320b4d64868e677d0ff8d552/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.461554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c170d396473c44f48dea5e372c055ba0, user_id=21be934b320b4d64868e677d0ff8d552, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.466134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.505052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 968/1934] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:54 2022] PUT /identity/v3/projects/c170d396473c44f48dea5e372c055ba0/users/21be934b320b4d64868e677d0ff8d552/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:54.507516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.515305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.542795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 967/1935] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:54.544487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.552958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.552958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.596389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] MFA Rules not processed for user `21be934b320b4d64868e677d0ff8d552`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:54.598031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.627535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dddee6-216e-476d-9fd2-5f617d980f27 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.635544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.636923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.637354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.637780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] PATH_INFO: `/v3/projects/c546091e7fdb4b92ac933f3a2d8fa321` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.652561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c546091e7fdb4b92ac933f3a2d8fa321)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.657815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.741261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:54.743074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c546091e7fdb4b92ac933f3a2d8fa321', 'request_id': 'req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.754758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c546091e7fdb4b92ac933f3a2d8fa321', 'request_id': 'req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.762827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c546091e7fdb4b92ac933f3a2d8fa321', 'request_id': 'req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.775167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 968/1936] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:54.777984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.780513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.782228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] The token cache is being invalidate because project c546091e7fdb4b92ac933f3a2d8fa321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:54.782359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.783710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.785488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 969/1937] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:54 2022] DELETE /identity/v3/projects/c546091e7fdb4b92ac933f3a2d8fa321 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:54.786371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.788692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.864601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a33bf8-6b07-4d84-bb12-c67cb50cddcd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.876483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca8e2-9eb5-40d3-80a6-a3248014f7a5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:54.881802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.882815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.883101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.883392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] PATH_INFO: `/v3/projects/02f607a953e94bc6814474ac8bfd25f1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.884753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f607a953e94bc6814474ac8bfd25f1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:54.886835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:54.927557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:54.928478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:54.928478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:54.929417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:54.930872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:54.952566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f607a953e94bc6814474ac8bfd25f1', 'request_id': 'req-104bb3b9-98a2-48cf-9dce-b42adf183726'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.961736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f607a953e94bc6814474ac8bfd25f1', 'request_id': 'req-104bb3b9-98a2-48cf-9dce-b42adf183726'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.962150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:54.970284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f607a953e94bc6814474ac8bfd25f1', 'request_id': 'req-104bb3b9-98a2-48cf-9dce-b42adf183726'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.981533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] The token cache is being invalidate because project 02f607a953e94bc6814474ac8bfd25f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:54.981828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.982557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:54.984336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 970/1938] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:54 2022] DELETE /identity/v3/projects/02f607a953e94bc6814474ac8bfd25f1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:54.985271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:54.987555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.012953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.015251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.025918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 969/1939] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:55.026761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.030062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.042357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207af561-4977-4bdf-a44e-607b43d6d2a1 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.072499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb3b9-98a2-48cf-9dce-b42adf183726 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.078964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:55.079603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.079896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.080146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] PATH_INFO: `/v3/users/ff14869177b348e1816aa1dcdebfb5db` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.081662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff14869177b348e1816aa1dcdebfb5db)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.083433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.089859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:55.131546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:55.132550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.132982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.133143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.136720 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.163016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff14869177b348e1816aa1dcdebfb5db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.164846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff14869177b348e1816aa1dcdebfb5db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.166313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff14869177b348e1816aa1dcdebfb5db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.193290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff14869177b348e1816aa1dcdebfb5db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.207960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cc3db372-125f-4871-a144-b9b437dfc4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff14869177b348e1816aa1dcdebfb5db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.242215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.247937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.265895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 971/1940] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:54 2022] DELETE /identity/v3/users/ff14869177b348e1816aa1dcdebfb5db => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:55.267816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.268235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.268644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.271503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 970/1941] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:55.272307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.273976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.302825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cebc9-52f0-4943-b3d7-1523c2f6067b service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.310549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:55.312184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.395895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 972/1942] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:55.396723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.407963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.452871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:55.453639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.454015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.455518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] PATH_INFO: `/v3/users/6a1edb063abb419b8618e763558db7a9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.457866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1edb063abb419b8618e763558db7a9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.460285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.481432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:55.502833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07415b66-bb48-46bb-81b0-1476fe52f892 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.508830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:55.509910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.510196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.510562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.514610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.535927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1edb063abb419b8618e763558db7a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.582555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1edb063abb419b8618e763558db7a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.594132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1edb063abb419b8618e763558db7a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.595369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1edb063abb419b8618e763558db7a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.596435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-684b597f-d133-4522-aeca-2bdd69e4fc05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1edb063abb419b8618e763558db7a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:55.596994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:55.621580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 971/1943] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:55 2022] DELETE /identity/v3/users/6a1edb063abb419b8618e763558db7a9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:55.624018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.624504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.625045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.649929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.653120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011d33-e1fe-40c7-9d45-e88b2735ef34 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.662263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:55.665575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.679623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 973/1944] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:55.681339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.681923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.682207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.708388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:55.709360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:55.779084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 972/1945] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:55.783414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.796446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d0bc5-d6d8-4010-9b60-bbdf7422bf18 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.805945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:55.806580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.807392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.807681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.808394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:55.808704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.810926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.828013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 973/1946] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:16:55.832054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.840421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 974/1947] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:55.843676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846bcb9-0d87-48be-b327-4299479d346d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.848524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:55.849204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:55.849414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:55.849693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:55.850334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:55.852289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:55.888350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 974/1948] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:55 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:55.895395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.909763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c625d-dd50-4b6b-a2dd-fb7965d20b6d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:55.930875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:55.933550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:56.162007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:56.162639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.162982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.163260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.163960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:56.168717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:56.194360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c7049-bee8-4964-8bda-00d1e2020ed3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:56.205290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:56.206154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.206544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.207500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.218132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 975/1949] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:55 2022] POST /identity/v3/users => generated 292 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:56.223110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:56.234016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:56.240091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:56.259061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587673ab-1204-44a5-97df-66e5a908eb95 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:56.262085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:56.271803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:56.275844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.276289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.276571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.277525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 975/1950] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:56.277991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:56.280267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:56.301275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 976/1951] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:56.302672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:56.321032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170501d2-dd81-4645-ba58-579eb0025554 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:56.329391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28137090-6ca1-4148-9383-db3c3ae501d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:56.330424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28137090-6ca1-4148-9383-db3c3ae501d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.330717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28137090-6ca1-4148-9383-db3c3ae501d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.331122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28137090-6ca1-4148-9383-db3c3ae501d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.332154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28137090-6ca1-4148-9383-db3c3ae501d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:56.333969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28137090-6ca1-4148-9383-db3c3ae501d5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:56.356346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:56.364460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 976/1952] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:56.374062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002384-9c94-4719-8625-afae56d37c5f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:56.381140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:56.381856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.382191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.382467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] PATH_INFO: `/v3/projects/57cd54d9d3e7493cb82918e73f8626df/users/2a1fe96a0b944fdc9995fecd4fb7d266/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.400467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cd54d9d3e7493cb82918e73f8626df, user_id=2a1fe96a0b944fdc9995fecd4fb7d266, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:56.403249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:56.418546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 977/1953] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:56 2022] PUT /identity/v3/projects/57cd54d9d3e7493cb82918e73f8626df/users/2a1fe96a0b944fdc9995fecd4fb7d266/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:56.429566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.429907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.430396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.474258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] MFA Rules not processed for user `2a1fe96a0b944fdc9995fecd4fb7d266`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:56.476419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:56.603886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 977/1954] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:56.610577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:56.612341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:56.679195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:56.712275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac4bf3-0eb9-418b-9886-78ffc4c6c8ae admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:56.718995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:56.719767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:56.720054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:56.720365 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:56.730583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:56.732520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:56.754300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 978/1955] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:56.917757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:56.919217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:57.020072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4dd4-9d5e-4db3-a541-0ff9a983aab2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:57.031823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:57.032957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.033466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.036507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] PATH_INFO: `/v3/projects/a78cdef84e444f8d9e4991a8281f4c85` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.038682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78cdef84e444f8d9e4991a8281f4c85)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:57.041232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:57.156346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78cdef84e444f8d9e4991a8281f4c85', 'request_id': 'req-c9a40bed-1af1-4d8f-a940-10567aceb24c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.166767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:57.168182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78cdef84e444f8d9e4991a8281f4c85', 'request_id': 'req-c9a40bed-1af1-4d8f-a940-10567aceb24c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.168566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:57.187692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78cdef84e444f8d9e4991a8281f4c85', 'request_id': 'req-c9a40bed-1af1-4d8f-a940-10567aceb24c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.207583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c707c-52cd-4a8b-80b3-7d1604af09b2 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:57.230638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] The token cache is being invalidate because project a78cdef84e444f8d9e4991a8281f4c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:57.231736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.233578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c9a40bed-1af1-4d8f-a940-10567aceb24c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.244354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 978/1956] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:56 2022] DELETE /identity/v3/projects/a78cdef84e444f8d9e4991a8281f4c85 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:57.247133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6945a-434c-46fc-ae03-950978d80386 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.247726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6945a-434c-46fc-ae03-950978d80386 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.248855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6945a-434c-46fc-ae03-950978d80386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.286317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a8a6945a-434c-46fc-ae03-950978d80386 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:57.289173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a8a6945a-434c-46fc-ae03-950978d80386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:57.297858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:57.332847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:57.333637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.333982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.334320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] PATH_INFO: `/v3/projects/22faf57641344f809c2aaacb1afeb193` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.336395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22faf57641344f809c2aaacb1afeb193)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:57.342888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:57.449545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22faf57641344f809c2aaacb1afeb193', 'request_id': 'req-2d07ab79-a582-4885-b300-472bcc9d5e52'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.466188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 979/1957] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:57.475000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22faf57641344f809c2aaacb1afeb193', 'request_id': 'req-2d07ab79-a582-4885-b300-472bcc9d5e52'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.475831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.476173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.476493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.485116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22faf57641344f809c2aaacb1afeb193', 'request_id': 'req-2d07ab79-a582-4885-b300-472bcc9d5e52'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.503950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:57.505032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] The token cache is being invalidate because project 22faf57641344f809c2aaacb1afeb193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:57.505228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:57.506596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.507864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.511469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 979/1958] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:57 2022] DELETE /identity/v3/projects/22faf57641344f809c2aaacb1afeb193 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:57.518005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:57.521657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:57.596242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 980/1959] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:57.599412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:57.603084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07ab79-a582-4885-b300-472bcc9d5e52 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:57.610781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:57.611436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.611818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.611936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df03c3-c37e-4215-9be6-df5afbebe3ee None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:57.612675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] PATH_INFO: `/v3/users/3fdeb3d570f04669b98c8adbd49c4567` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.613683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fdeb3d570f04669b98c8adbd49c4567)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:57.615733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:57.617003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:57.617815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.618047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.618328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.619039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:57.619359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:57.621196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:57.636081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:57.660085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 981/1960] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:16:57.664663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:57.699634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0c3c-82bc-4859-a950-e63bc51f8145 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:57.709539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:57.711498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:57.711979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:57.712289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:57.712913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:57.714067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fdeb3d570f04669b98c8adbd49c4567'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.715114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fdeb3d570f04669b98c8adbd49c4567'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.715506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:57.716644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fdeb3d570f04669b98c8adbd49c4567'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.733971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fdeb3d570f04669b98c8adbd49c4567'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.752832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fdeb3d570f04669b98c8adbd49c4567'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:57.777488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 980/1961] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:57 2022] DELETE /identity/v3/users/3fdeb3d570f04669b98c8adbd49c4567 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:57.791820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 982/1962] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:57 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:57.798679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:57.803271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:57.924535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38ce4a-f6ae-4a15-bd64-209ff7a62397 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.033655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.034284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.034615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.034894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.035600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.047386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.085996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 981/1963] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:57 2022] POST /identity/v3/users => generated 287 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:58.090004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.102783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd727d-45f0-4219-a94d-1815e0e18a86 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.109379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.110010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.110698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.110698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.111198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.112867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.128109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 983/1964] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:16:58.129026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.140316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08731a-db3a-4642-8371-777009d21272 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.148031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.148695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.149016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.149241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.149939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.151556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.164349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 982/1965] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:58.165790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.167782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:58.170028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.183639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2b593-35b6-4f29-963e-7f2a612fefa6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.183767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8586dd-26c1-4c34-a3b7-3ef5ae7a35e7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.199528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c4571-f2c1-42ff-99af-cd54ab6eff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.200527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.200707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c4571-f2c1-42ff-99af-cd54ab6eff5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.200993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c4571-f2c1-42ff-99af-cd54ab6eff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.201333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c4571-f2c1-42ff-99af-cd54ab6eff5c admin admin] PATH_INFO: `/v3/projects/46ab441b7dcd4611a77ed792c4bc8a9f/users/c6f76402e6574466b0df7ca1dde62377/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.202131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.202824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.203315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] PATH_INFO: `/v3/projects/3ee68f858a84444f8b657e95f4702566` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.204832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee68f858a84444f8b657e95f4702566)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.207266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.222407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:58.228664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c4571-f2c1-42ff-99af-cd54ab6eff5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ab441b7dcd4611a77ed792c4bc8a9f, user_id=c6f76402e6574466b0df7ca1dde62377, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.231875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c4571-f2c1-42ff-99af-cd54ab6eff5c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.252227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 983/1966] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:58 2022] PUT /identity/v3/projects/46ab441b7dcd4611a77ed792c4bc8a9f/users/c6f76402e6574466b0df7ca1dde62377/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:58.255819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.256099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.256435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.282737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:58.284223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:58.287068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee68f858a84444f8b657e95f4702566', 'request_id': 'req-e57aa848-e826-4f2f-860a-89e3f87eb09f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:58.296827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee68f858a84444f8b657e95f4702566', 'request_id': 'req-e57aa848-e826-4f2f-860a-89e3f87eb09f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:58.299673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:58.309814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee68f858a84444f8b657e95f4702566', 'request_id': 'req-e57aa848-e826-4f2f-860a-89e3f87eb09f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:58.335655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] The token cache is being invalidate because project 3ee68f858a84444f8b657e95f4702566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:16:58.336126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:58.336929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e57aa848-e826-4f2f-860a-89e3f87eb09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:58.339360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 984/1967] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:16:58 2022] DELETE /identity/v3/projects/3ee68f858a84444f8b657e95f4702566 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:58.346398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a2a23-7e87-41a0-b2b0-2ce395577e28 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.347251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a2a23-7e87-41a0-b2b0-2ce395577e28 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.347971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a2a23-7e87-41a0-b2b0-2ce395577e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.358522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 984/1968] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:58.365105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.366591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:58.380044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bf2a2a23-7e87-41a0-b2b0-2ce395577e28 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:58.381410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bf2a2a23-7e87-41a0-b2b0-2ce395577e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:58.390976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 985/1969] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:58.399747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.400122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.400498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.428126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:58.429415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:58.438076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66705a-40d6-4201-9d6d-57f11eaf812a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.439026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 986/1970] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:16:58.442068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.442801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.443140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.443439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.443606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.457872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e48a7-d9fe-43ed-bfc6-92630603de98 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.459895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.461514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.471392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.472378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.473199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.473199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.475067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:16:58.475382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.479042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.489782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 985/1971] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:16:58.507701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 987/1972] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:16:58.512664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.531105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7a1bb-a8cd-4890-aee2-abbf7227dbeb service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.556795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.557435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.558922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.559245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.559905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.573729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.632345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 986/1973] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:58 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:58.637489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.650354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aeadd1-dec3-4b4b-a59e-444c7736acbf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.754839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.755885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.755885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.756065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.756714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.758730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.814141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 988/1974] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:58 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:16:58.817940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.842336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb871-8507-4ea1-828e-d61acbcda411 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.856986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.860296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.864725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.865054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.866819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.869766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.895707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 987/1975] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:58.897973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:58.918004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838495-24dd-48e0-9e6c-f51469f802df admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:58.933780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38710-5dc5-4ef4-a506-9530f16f67a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:58.934417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38710-5dc5-4ef4-a506-9530f16f67a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:58.934980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38710-5dc5-4ef4-a506-9530f16f67a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:58.935781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38710-5dc5-4ef4-a506-9530f16f67a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:58.936668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b38710-5dc5-4ef4-a506-9530f16f67a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:58.938459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b38710-5dc5-4ef4-a506-9530f16f67a6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:58.971200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 989/1976] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:58.980051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.012713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a864-8fbb-44cc-8897-8481913e3f70 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.067470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.073396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.073396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.073396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] PATH_INFO: `/v3/projects/c6b85ee61646400f810c32456df8409f/users/d8e2f5469e324822ab5bf6ff8777ce2e/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.113356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b85ee61646400f810c32456df8409f, user_id=d8e2f5469e324822ab5bf6ff8777ce2e, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.118774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.144645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 988/1977] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:58 2022] PUT /identity/v3/projects/c6b85ee61646400f810c32456df8409f/users/d8e2f5469e324822ab5bf6ff8777ce2e/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:59.155142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.155484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.155763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.232739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.234550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.236995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] MFA Rules not processed for user `d8e2f5469e324822ab5bf6ff8777ce2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:59.238123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.331094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:59.387014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea99dae-fe62-4d5a-97a7-819a8f98cdee admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.398780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.399946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.400436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.401067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] PATH_INFO: `/v3/users/37fa87c3377a4cafb1b24ce2ba37a348` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.402031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 990/1978] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:59.408374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fa87c3377a4cafb1b24ce2ba37a348)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.409346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.422259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.435468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7363523-ab29-4450-b899-8ecf701ebf87 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.442567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417f589-7f41-41aa-8c66-0763680b3b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.443144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417f589-7f41-41aa-8c66-0763680b3b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.443511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417f589-7f41-41aa-8c66-0763680b3b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.443950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417f589-7f41-41aa-8c66-0763680b3b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.445448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417f589-7f41-41aa-8c66-0763680b3b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.447071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417f589-7f41-41aa-8c66-0763680b3b9a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.500083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 991/1979] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:59.501514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510e2f-bad7-440e-a579-f05791b2e2ef None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.501995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510e2f-bad7-440e-a579-f05791b2e2ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.502289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510e2f-bad7-440e-a579-f05791b2e2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.513045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fa87c3377a4cafb1b24ce2ba37a348'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:59.514271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fa87c3377a4cafb1b24ce2ba37a348'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:59.515270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fa87c3377a4cafb1b24ce2ba37a348'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:59.522153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:59.546239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fa87c3377a4cafb1b24ce2ba37a348'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:59.557404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-45510e2f-bad7-440e-a579-f05791b2e2ef None None] MFA Rules not processed for user `c6f76402e6574466b0df7ca1dde62377`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:59.558684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-45510e2f-bad7-440e-a579-f05791b2e2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.562374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fa87c3377a4cafb1b24ce2ba37a348'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:16:59.580521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 989/1980] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:16:59 2022] DELETE /identity/v3/users/37fa87c3377a4cafb1b24ce2ba37a348 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:16:59.581170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.583053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.663926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 992/1981] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:59.665297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.665507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.665779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.670312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2722807-a432-4f5d-8547-2b337c5eb974 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.675438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.676057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.676414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.676693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.677321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.679246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.686027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:16:59.687256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.695668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 993/1982] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:16:59.696290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.697529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.707550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 990/1983] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:59.708473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.714488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf910-296f-405e-8af3-462b82258a9e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.719176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8637cc-96b8-4878-a6a1-4bb6955af7da admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.719827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.720515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.720826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.721098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.721726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.723648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.733420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.734055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.734335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.734637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.735295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.736947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.749586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 991/1984] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:16:59.750344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.752137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:16:59.770926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 994/1985] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:16:59.771755 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.804801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:59.839103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f50fa-44c2-41ca-ab8b-6a026d93f7c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.865489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007e21-28c2-47ff-87ae-2a99cac7cbf5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.871025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.871681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.871951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.872290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] PATH_INFO: `/v3/projects/733fc9abe4094cdfabe5774dc8dae73b/users/65a98edac6b148858c93763a414506af/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.890966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733fc9abe4094cdfabe5774dc8dae73b, user_id=65a98edac6b148858c93763a414506af, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.894009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.902737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:16:59.903368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:16:59.903702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:16:59.904610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:16:59.910632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 995/1986] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:16:59 2022] PUT /identity/v3/projects/733fc9abe4094cdfabe5774dc8dae73b/users/65a98edac6b148858c93763a414506af/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:16:59.911535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.921668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:16:59.923148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:16:59.933558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 992/1987] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:16:59.934401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:16:59.945490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98d8ff-4475-4f9c-ac12-acd0a3742088 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:16:59.974904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:16:59.996555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb865b80-e63c-4994-9d8d-8304378d7020 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.003980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.005553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.005553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.005917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.006562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.009317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.034733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.035508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.035793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.036105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.037530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.039099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.051601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 996/1988] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:16:59 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:00.053192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 993/1989] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:00.055901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.058995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.072041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f7f5e-4bb9-4a58-a702-935eba1a86dd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.080419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.080864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18f12-3d94-43f1-99d8-078555cbb8c2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.081351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.081701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.082505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.084536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.088902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc15f2-e128-4f0c-a631-44c6501da290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.089397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.089982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc15f2-e128-4f0c-a631-44c6501da290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.090431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc15f2-e128-4f0c-a631-44c6501da290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.090894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc15f2-e128-4f0c-a631-44c6501da290 admin admin] PATH_INFO: `/v3/projects/733fc9abe4094cdfabe5774dc8dae73b/users/65a98edac6b148858c93763a414506af/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.094547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc15f2-e128-4f0c-a631-44c6501da290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733fc9abe4094cdfabe5774dc8dae73b, user_id=65a98edac6b148858c93763a414506af, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.099667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc15f2-e128-4f0c-a631-44c6501da290 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.112561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 997/1990] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:00.113385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.125003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139f9e-3bd6-4439-8253-383db4984f54 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.128289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 994/1991] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:00 2022] PUT /identity/v3/projects/733fc9abe4094cdfabe5774dc8dae73b/users/65a98edac6b148858c93763a414506af/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:00.145080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.145586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.146035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.200310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.200996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] MFA Rules not processed for user `65a98edac6b148858c93763a414506af`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:00.201773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.201773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.202263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] PATH_INFO: `/v3/projects/992c975ce7194088837e4f12aba0c0dd/users/ca5dbf32e4f946e794bf90469651f78f/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.203249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:00.218767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=992c975ce7194088837e4f12aba0c0dd, user_id=ca5dbf32e4f946e794bf90469651f78f, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.221851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.238584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 998/1992] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:00 2022] PUT /identity/v3/projects/992c975ce7194088837e4f12aba0c0dd/users/ca5dbf32e4f946e794bf90469651f78f/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:00.242431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.293263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:00.308572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 995/1993] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:00.321763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.323760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:00.339064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eca20f-6a43-4f52-9e44-c8eca7288ea4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.344535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.345301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.345589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.345866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.346522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.348142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.362079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 999/1994] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:00.365990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.385355 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845b77e-dee4-4c2b-bf22-37ac73752825 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.394057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.394839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.395219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.395602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] PATH_INFO: `/v3/projects/992c975ce7194088837e4f12aba0c0dd/users/ca5dbf32e4f946e794bf90469651f78f/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.397932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=992c975ce7194088837e4f12aba0c0dd, user_id=ca5dbf32e4f946e794bf90469651f78f, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.404274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.405572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ededb1-1a38-4ff8-887a-ac99fb95682f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.427107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1000/1995] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:00 2022] PUT /identity/v3/projects/992c975ce7194088837e4f12aba0c0dd/users/ca5dbf32e4f946e794bf90469651f78f/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:00.492952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.494459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.496229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.496229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.550052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.551949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:00.617837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.619806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.633028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:00.649087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 996/1996] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:00.714862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb5321-7ec9-4ac7-86f3-ded011da6318 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.725660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.726836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.727123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.727399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] PATH_INFO: `/v3/projects/6be6791ac9da47afb8a5fb18392b31f2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.729066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be6791ac9da47afb8a5fb18392b31f2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.731480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.784671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.797953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a5ccb-d6d0-4ec6-94e7-5e60685bfd76 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.806713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be6791ac9da47afb8a5fb18392b31f2', 'request_id': 'req-c96d504e-6e4a-44d3-9868-e2c82272a816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.817091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be6791ac9da47afb8a5fb18392b31f2', 'request_id': 'req-c96d504e-6e4a-44d3-9868-e2c82272a816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.829001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be6791ac9da47afb8a5fb18392b31f2', 'request_id': 'req-c96d504e-6e4a-44d3-9868-e2c82272a816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.844345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] The token cache is being invalidate because project 6be6791ac9da47afb8a5fb18392b31f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:00.844831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.846346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.847785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1001/1997] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:00 2022] DELETE /identity/v3/projects/6be6791ac9da47afb8a5fb18392b31f2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:00.851746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:00.853252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:00.874265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.875071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.875185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.875445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] PATH_INFO: `/v3/users/d8e2f5469e324822ab5bf6ff8777ce2e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.876808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e2f5469e324822ab5bf6ff8777ce2e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.879095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.885568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:00.943190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d504e-6e4a-44d3-9868-e2c82272a816 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:00.949516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:00.950211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:00.950495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:00.950909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] PATH_INFO: `/v3/users/09adf9fc45144607ba068a83e2c80186` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:00.952715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09adf9fc45144607ba068a83e2c80186)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:00.954877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:00.958359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e2f5469e324822ab5bf6ff8777ce2e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.959307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e2f5469e324822ab5bf6ff8777ce2e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.960402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e2f5469e324822ab5bf6ff8777ce2e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.979922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e2f5469e324822ab5bf6ff8777ce2e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:00.995121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a0dcfd4-2870-4bdf-925f-4c6c9cbd401f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e2f5469e324822ab5bf6ff8777ce2e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.018322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 997/1998] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:00 2022] DELETE /identity/v3/users/d8e2f5469e324822ab5bf6ff8777ce2e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:01.032987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa3f7-e3f3-4ed0-9dd6-eb2276f30009 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:01.033691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa3f7-e3f3-4ed0-9dd6-eb2276f30009 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:01.034036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09adf9fc45144607ba068a83e2c80186'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.034455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa3f7-e3f3-4ed0-9dd6-eb2276f30009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:01.064787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09adf9fc45144607ba068a83e2c80186'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.065094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-da8fa3f7-e3f3-4ed0-9dd6-eb2276f30009 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:01.069232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-da8fa3f7-e3f3-4ed0-9dd6-eb2276f30009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.069730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:01.081961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09adf9fc45144607ba068a83e2c80186'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.082621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09adf9fc45144607ba068a83e2c80186'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.083522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09adf9fc45144607ba068a83e2c80186'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.108361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1002/1999] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:00 2022] DELETE /identity/v3/users/09adf9fc45144607ba068a83e2c80186 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:01.152607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 998/2000] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:01.165646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:01.167456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.234852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198aafd-fa49-4403-a237-6fbebc0d22d5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:01.239226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:01.239663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:01.240008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:01.240686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:01.348955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:01.349392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:01.349679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:01.386834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] MFA Rules not processed for user `ca5dbf32e4f946e794bf90469651f78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:01.388789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.390108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:01.396465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:01.427505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1003/2001] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:01.496245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:01.498347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.532481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 999/2002] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:01.533769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:01.535239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.539451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3093b3-dc29-4662-a5ee-067c740e9f05 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:01.549365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:01.550038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:01.550322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:01.550595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] PATH_INFO: `/v3/projects/9f299c8eb7a549c0841a1ca425995a19` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:01.551834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:01.553060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f299c8eb7a549c0841a1ca425995a19)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:01.554985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:01.626290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96cec9-58c8-40b1-ab38-583eed37a376 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:01.634555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:01.636132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:01.636434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:01.636704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:01.640981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.663100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f299c8eb7a549c0841a1ca425995a19', 'request_id': 'req-bcfbaad6-222d-4df5-9a91-2b686b460f91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.674242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f299c8eb7a549c0841a1ca425995a19', 'request_id': 'req-bcfbaad6-222d-4df5-9a91-2b686b460f91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.687780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f299c8eb7a549c0841a1ca425995a19', 'request_id': 'req-bcfbaad6-222d-4df5-9a91-2b686b460f91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.705362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] The token cache is being invalidate because project 9f299c8eb7a549c0841a1ca425995a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:01.705666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.706956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.714401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1004/2003] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:01 2022] DELETE /identity/v3/projects/9f299c8eb7a549c0841a1ca425995a19 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:01.718631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:01.720359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:01.751765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:01.760483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:01.762458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:01.785541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1000/2004] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:01.825297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbaad6-222d-4df5-9a91-2b686b460f91 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:01.833077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:01.833898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:01.834283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:01.834861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] PATH_INFO: `/v3/users/21be934b320b4d64868e677d0ff8d552` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:01.836740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21be934b320b4d64868e677d0ff8d552)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:01.839176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:01.939407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21be934b320b4d64868e677d0ff8d552'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.966830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21be934b320b4d64868e677d0ff8d552'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.984849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21be934b320b4d64868e677d0ff8d552'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.985837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21be934b320b4d64868e677d0ff8d552'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:01.986507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21be934b320b4d64868e677d0ff8d552'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.012570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1005/2005] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:01 2022] DELETE /identity/v3/users/21be934b320b4d64868e677d0ff8d552 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:02.227732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:02.229451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:02.261494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2cc66-dff7-4c91-a764-2f384bfc884e service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:02.391741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:02.392461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.392839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.393047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] PATH_INFO: `/v3/projects/c6b85ee61646400f810c32456df8409f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.394274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b85ee61646400f810c32456df8409f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:02.396183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:02.451254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b85ee61646400f810c32456df8409f', 'request_id': 'req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.459699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b85ee61646400f810c32456df8409f', 'request_id': 'req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.467300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b85ee61646400f810c32456df8409f', 'request_id': 'req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.473310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:02.483492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] The token cache is being invalidate because project c6b85ee61646400f810c32456df8409f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:02.483910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.484652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.486908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1001/2006] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:02 2022] DELETE /identity/v3/projects/c6b85ee61646400f810c32456df8409f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:02.492773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:02.492931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:02.494376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:02.496255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:02.519316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3ec8f-0b28-4ca1-a8af-2fdaaa400623 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:02.570845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799aa3e-7591-4fb3-b192-f7ecdb48a89f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:02.575523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:02.576394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.576760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.576949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] PATH_INFO: `/v3/projects/c170d396473c44f48dea5e372c055ba0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.578108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c170d396473c44f48dea5e372c055ba0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:02.580056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:02.584008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:02.584400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.584691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.585055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] PATH_INFO: `/v3/users/65a98edac6b148858c93763a414506af` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.586132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a98edac6b148858c93763a414506af)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:02.588147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:02.631828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:02.632177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c170d396473c44f48dea5e372c055ba0', 'request_id': 'req-5429ee97-4841-476f-83d3-828127ac5ed8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.639556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a98edac6b148858c93763a414506af'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.643294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c170d396473c44f48dea5e372c055ba0', 'request_id': 'req-5429ee97-4841-476f-83d3-828127ac5ed8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.650600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c170d396473c44f48dea5e372c055ba0', 'request_id': 'req-5429ee97-4841-476f-83d3-828127ac5ed8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.656635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a98edac6b148858c93763a414506af'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.662896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] The token cache is being invalidate because project c170d396473c44f48dea5e372c055ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:02.663204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.663921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5429ee97-4841-476f-83d3-828127ac5ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.670558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1002/2007] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:02 2022] DELETE /identity/v3/projects/c170d396473c44f48dea5e372c055ba0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:02.670730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a98edac6b148858c93763a414506af'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.670730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a98edac6b148858c93763a414506af'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.670730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ecdd183f-7c5f-41d1-88c7-19ef39fc0875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a98edac6b148858c93763a414506af'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:02.672432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.672574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.672793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.684404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1006/2008] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:02 2022] DELETE /identity/v3/users/65a98edac6b148858c93763a414506af => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:02.696661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:02.697895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:02.775266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1003/2009] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:02.782317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.782836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.783118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.803906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:02.805173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:02.813955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1007/2010] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:02.818289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:02.828433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8280eae-3623-4e36-a069-d05f97475708 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:02.833299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:02.834026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.834518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.834635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.835241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:02.835543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:02.837232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:02.851042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1004/2011] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:02.855462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:02.866458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796519e-2f2c-40b0-8f50-31893489b45c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:02.871927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:02.872653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:02.872930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:02.873198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:02.873891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:02.876380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:02.914678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1008/2012] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:02 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:02.919205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:02.931198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a70ac-3a82-4086-a042-516d1b3483c3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:02.944356 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:03.017122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.018188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.018643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.019081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.020097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.022570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.052324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1005/2013] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:02 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:03.057657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.077733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128335-6ac4-4f1b-bf5b-d7bb4c99cd2c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.084737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.085620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.086653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.086972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:03.087200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.087615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.088918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.092722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.102202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1aa13-361d-493b-b450-b14be6adeb70 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.110228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.110681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.110961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.111229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] PATH_INFO: `/v3/projects/733fc9abe4094cdfabe5774dc8dae73b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.112417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=733fc9abe4094cdfabe5774dc8dae73b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.115832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1009/2014] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:03.115973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.117410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.137419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f8ea3-a7ae-4d75-a6bf-913144703ff9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.146647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.147667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.148231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.148717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.149901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.153028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.176399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1010/2015] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:03.179026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.191178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733fc9abe4094cdfabe5774dc8dae73b', 'request_id': 'req-226e897f-5c85-49c8-a4d5-36b1c8ffa618'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.203536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733fc9abe4094cdfabe5774dc8dae73b', 'request_id': 'req-226e897f-5c85-49c8-a4d5-36b1c8ffa618'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.209210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0b7b2-2601-4ed2-aa04-92ed7e366b25 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.212441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733fc9abe4094cdfabe5774dc8dae73b', 'request_id': 'req-226e897f-5c85-49c8-a4d5-36b1c8ffa618'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.227940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] The token cache is being invalidate because project 733fc9abe4094cdfabe5774dc8dae73b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:03.228305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.229003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-226e897f-5c85-49c8-a4d5-36b1c8ffa618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.230994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1006/2016] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:03 2022] DELETE /identity/v3/projects/733fc9abe4094cdfabe5774dc8dae73b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:03.244882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d47c-a71c-4a8b-bd39-cb420dd82a8a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.244882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d47c-a71c-4a8b-bd39-cb420dd82a8a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.244882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d47c-a71c-4a8b-bd39-cb420dd82a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.274090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-30a1d47c-a71c-4a8b-bd39-cb420dd82a8a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:03.275418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30a1d47c-a71c-4a8b-bd39-cb420dd82a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:03.314498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b519fc8-3ef2-46f6-93c2-94238bde302b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.315567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519fc8-3ef2-46f6-93c2-94238bde302b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.316010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519fc8-3ef2-46f6-93c2-94238bde302b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.316626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b519fc8-3ef2-46f6-93c2-94238bde302b admin admin] PATH_INFO: `/v3/projects/012dcf0d2246498abacf0ba93776f9fe/users/c28ce0f391b549c2adb1651ccc0b2e1d/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.357386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519fc8-3ef2-46f6-93c2-94238bde302b admin admin] RBAC: Authorizing `identity:create_grant(project_id=012dcf0d2246498abacf0ba93776f9fe, user_id=c28ce0f391b549c2adb1651ccc0b2e1d, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.363296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b519fc8-3ef2-46f6-93c2-94238bde302b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.392374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1011/2017] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:03 2022] PUT /identity/v3/projects/012dcf0d2246498abacf0ba93776f9fe/users/c28ce0f391b549c2adb1651ccc0b2e1d/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:03.403761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.404124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.404438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.408997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1007/2018] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:03.418099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.418647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.419134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.448993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:03.450799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:03.473037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] MFA Rules not processed for user `c28ce0f391b549c2adb1651ccc0b2e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:03.476584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:03.492228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:03.543489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:03.569161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1008/2019] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:03.570901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.581199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7746bee-2fc8-42b5-85e5-6b74b4d369f7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.586786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.590976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1012/2020] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:03.591157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.591157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.591157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.591157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:03.591157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.591614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.602561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.604264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:03.606819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1009/2021] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:03.611781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.622347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045634d-3928-409f-b7fd-7645dac5ce77 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.628604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.629334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.629714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.630095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.630849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.633722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.675469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1010/2022] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:03 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:03.677472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089b561-7399-47f3-b128-a43c302c6e48 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.679481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.730056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.730673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.730956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.731226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.758078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15900653-6112-4e7d-b062-0f2771cc20d5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.763048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.764348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.764801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.765073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.765695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.767862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.793580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1011/2023] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:03 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:03.797710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.807731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.809225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.809355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0300e2c-ea94-4ed6-9a61-a8a11c94fb60 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.814614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.815310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.815668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.815947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.816633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.818467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.821167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1013/2024] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:03.832677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1012/2025] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:03.834728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.849388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c778e-8082-4cd0-8e70-5e233c35a246 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.857158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.859330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:03.860965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.862134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.862134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.862425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.863005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.864863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.874580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bd83-caba-48ba-9b7f-5df269411e32 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.878090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1014/2026] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:03.883409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.883938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:03.884093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.884364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.884631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] PATH_INFO: `/v3/users/c28ce0f391b549c2adb1651ccc0b2e1d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.885966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28ce0f391b549c2adb1651ccc0b2e1d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.887931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.898404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede926-2ff5-4eb5-8450-17faac081ee0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:03.910077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b4739-478f-4bcd-b5f3-042b9d6f346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:03.911097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b4739-478f-4bcd-b5f3-042b9d6f346e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.911573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b4739-478f-4bcd-b5f3-042b9d6f346e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.912100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b4739-478f-4bcd-b5f3-042b9d6f346e admin admin] PATH_INFO: `/v3/projects/0e0a690be6cd4c28aa9f38edb76bee0c/users/d1d3dccd729c464cb5d52abfe3d6e2e4/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.937873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b4739-478f-4bcd-b5f3-042b9d6f346e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0a690be6cd4c28aa9f38edb76bee0c, user_id=d1d3dccd729c464cb5d52abfe3d6e2e4, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:03.937873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b4739-478f-4bcd-b5f3-042b9d6f346e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:03.954704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1015/2027] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:03 2022] PUT /identity/v3/projects/0e0a690be6cd4c28aa9f38edb76bee0c/users/d1d3dccd729c464cb5d52abfe3d6e2e4/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:03.969081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:03.969081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:03.969081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:03.974953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28ce0f391b549c2adb1651ccc0b2e1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.975643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28ce0f391b549c2adb1651ccc0b2e1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.976987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28ce0f391b549c2adb1651ccc0b2e1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.994647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28ce0f391b549c2adb1651ccc0b2e1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:03.996617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:03.999098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:04.007140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] MFA Rules not processed for user `d1d3dccd729c464cb5d52abfe3d6e2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:04.010172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.015914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47e4e6e-bc01-4dd6-adb8-8b639586abc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28ce0f391b549c2adb1651ccc0b2e1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:04.031156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1013/2028] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:03 2022] DELETE /identity/v3/users/c28ce0f391b549c2adb1651ccc0b2e1d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:04.035602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.035893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.036171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.058126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:04.059375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.113609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1016/2029] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:04.121267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.128967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.143939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1014/2030] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:04.154608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.156646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.210085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f93599-65fc-4d45-8365-25f5554f228c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.221128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.221834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.222081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.222413 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.223045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.225018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.237946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2d1ea-3b42-4788-853b-c794e8e5fd83 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.242183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.242849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.243138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.243424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.259376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1017/2031] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:04 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:04.264074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.264536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.266931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.282316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1015/2032] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:04.375748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4cfb-12ac-4c2a-9316-3aff61e0f829 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.383404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.384184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.384758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.385260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.385940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.392547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.438934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1018/2033] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:04 2022] POST /identity/v3/users => generated 303 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:04.445519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.457114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253e330-2b12-4ab6-a0e2-f17a53abf362 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.462735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.462956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.464020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.464564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.464843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.465663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.466250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.468020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.486248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1016/2034] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:04.490971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.502373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:04.503396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebab2c-b11c-4e1c-a5d7-db2d75f81f7e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.514003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.516801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.517278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.518540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] PATH_INFO: `/v3/projects/d1c75e3109544cef8b373cbf0ef2b698/users/db2b3597e34845bcad4b94c96505691c/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.548438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c75e3109544cef8b373cbf0ef2b698, user_id=db2b3597e34845bcad4b94c96505691c, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.551976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb55cda-3c50-42f1-bbc2-aea4e7c78c74 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.553560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.557086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.558185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.558499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.558791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.561863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.580180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1017/2035] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:04 2022] PUT /identity/v3/projects/d1c75e3109544cef8b373cbf0ef2b698/users/db2b3597e34845bcad4b94c96505691c/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:04.584879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.671850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:04.682035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.686914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.700240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7947708-f1ca-4f87-934b-799f71d4cfbb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.706220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1019/2036] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:04.718481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30342c6-69dd-4b6b-951c-06f9c51934ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.719509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30342c6-69dd-4b6b-951c-06f9c51934ff admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.719720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30342c6-69dd-4b6b-951c-06f9c51934ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.720103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30342c6-69dd-4b6b-951c-06f9c51934ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.720691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30342c6-69dd-4b6b-951c-06f9c51934ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.726856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30342c6-69dd-4b6b-951c-06f9c51934ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.754132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1018/2037] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:04.758348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.772552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624d2c3-36e2-4ca2-8aa3-8e13295a3a24 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:04.777731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:04.778897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.779177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.779945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] PATH_INFO: `/v3/projects/d1c75e3109544cef8b373cbf0ef2b698/users/db2b3597e34845bcad4b94c96505691c/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.785295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c75e3109544cef8b373cbf0ef2b698, user_id=db2b3597e34845bcad4b94c96505691c, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:04.788791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:04.811373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1020/2038] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:04 2022] PUT /identity/v3/projects/d1c75e3109544cef8b373cbf0ef2b698/users/db2b3597e34845bcad4b94c96505691c/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:04.824366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:04.825575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:04.825969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:04.906443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] MFA Rules not processed for user `db2b3597e34845bcad4b94c96505691c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:04.907900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:04.929547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:04.931192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.035900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6cf1-dd69-400e-9474-398bdd050a63 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.044474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.048392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.048652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1019/2039] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:05.048781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.048781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] PATH_INFO: `/v3/projects/012dcf0d2246498abacf0ba93776f9fe` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.049200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] RBAC: Authorizing `identity:delete_project(project_id=012dcf0d2246498abacf0ba93776f9fe)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.059913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.066108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.116639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012dcf0d2246498abacf0ba93776f9fe', 'request_id': 'req-4f72d8da-ca3b-4bc5-9dc5-949944891add'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:05.121858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:05.124045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012dcf0d2246498abacf0ba93776f9fe', 'request_id': 'req-4f72d8da-ca3b-4bc5-9dc5-949944891add'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:05.124730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777115f-c7bd-420c-94e7-75b985dea5ae None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.135772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012dcf0d2246498abacf0ba93776f9fe', 'request_id': 'req-4f72d8da-ca3b-4bc5-9dc5-949944891add'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:05.152049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] The token cache is being invalidate because project 012dcf0d2246498abacf0ba93776f9fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:05.152503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:05.153209 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f72d8da-ca3b-4bc5-9dc5-949944891add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:05.155338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1021/2040] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:04 2022] DELETE /identity/v3/projects/012dcf0d2246498abacf0ba93776f9fe => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:05.164318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb3a72-ddde-4b37-8b02-a304d933bbed None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.164608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb3a72-ddde-4b37-8b02-a304d933bbed None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.164890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb3a72-ddde-4b37-8b02-a304d933bbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.183187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.183813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.184093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.184551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.186783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.187077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1ffb3a72-ddde-4b37-8b02-a304d933bbed None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:05.188287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1ffb3a72-ddde-4b37-8b02-a304d933bbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.240024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:05.271050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.272156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1022/2041] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:05.272531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.280936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.281218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.281488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.284126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1020/2042] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:05 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:05.303926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:05.305328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.313643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1023/2043] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:05.318076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.329289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fbbd2-46fc-4af1-96df-940ae054a70a service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.333879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.334474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.334751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.335020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.335653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:05.335948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.337568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.337752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.339097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.350829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1021/2044] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:05.351499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75429fcc-8b6a-4de4-9329-da688e207afc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.355051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.355245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.355867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.356122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.356417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.358284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.365043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c9ea-2b02-4b25-8ef1-cee164bdc1db admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.369855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.370377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.370657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.370927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.371601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.373558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.407040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1022/2045] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:05 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:05.411166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.423208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.424717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.435900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1024/2046] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:05.482380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e877ae-9ce1-41f9-9c69-4a0d229b6c4b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.488058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.488977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.489255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.489549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.490185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.492116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.516705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1023/2047] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:05 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:05.521062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.532556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f357-7a10-49e0-9728-3b1eb1563bd8 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.537647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.538249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.538526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.538788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.539416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.541167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.551979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1025/2048] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:05.556012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.567970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6763-ddf1-4bd9-b504-19c9dde6218b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.579810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ea8d7-485f-4e43-b84a-f91ac88ae771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.580481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ea8d7-485f-4e43-b84a-f91ac88ae771 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.580766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ea8d7-485f-4e43-b84a-f91ac88ae771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.581122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ea8d7-485f-4e43-b84a-f91ac88ae771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.581918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ea8d7-485f-4e43-b84a-f91ac88ae771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.583534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ea8d7-485f-4e43-b84a-f91ac88ae771 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.603434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1024/2049] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:05.604152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.617779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dff88b-dcc2-49fa-a5ca-32c0c06e5002 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:05.625525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:05.626366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.628277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.628778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] PATH_INFO: `/v3/projects/ede6199069114ce380f30088d312f345/users/7ea15f54d31149ea948650cdd363d42f/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.648456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede6199069114ce380f30088d312f345, user_id=7ea15f54d31149ea948650cdd363d42f, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:05.652039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:05.673084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1026/2050] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:05 2022] PUT /identity/v3/projects/ede6199069114ce380f30088d312f345/users/7ea15f54d31149ea948650cdd363d42f/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:05.679230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:05.679458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:05.679700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:05.721911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:05.725265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:05.914119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1025/2051] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:05.931448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:05.936992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:06.043373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:06.067822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1e88-bb2f-42ed-8819-8ecc5ef26c7a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:06.073095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60d44a-2c3f-4464-ad6b-dec33665496a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:06.074045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60d44a-2c3f-4464-ad6b-dec33665496a service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:06.074541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60d44a-2c3f-4464-ad6b-dec33665496a service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:06.075019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60d44a-2c3f-4464-ad6b-dec33665496a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:06.088070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60d44a-2c3f-4464-ad6b-dec33665496a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:06.089803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60d44a-2c3f-4464-ad6b-dec33665496a service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:06.102544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1027/2052] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:06.919889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:06.922035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:06.940532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db4f18-ec87-4e67-9e43-24a477ae5e4b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:06.954025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:06.954689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:06.954969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:06.955241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] PATH_INFO: `/v3/users/2a1fe96a0b944fdc9995fecd4fb7d266` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:06.959123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1fe96a0b944fdc9995fecd4fb7d266)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:06.961144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:07.001457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:07.128388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1fe96a0b944fdc9995fecd4fb7d266'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:07.129968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1fe96a0b944fdc9995fecd4fb7d266'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:07.131256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1fe96a0b944fdc9995fecd4fb7d266'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:07.167202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1fe96a0b944fdc9995fecd4fb7d266'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:07.189368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1fe96a0b944fdc9995fecd4fb7d266'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:07.215584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1026/2053] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:06 2022] DELETE /identity/v3/users/2a1fe96a0b944fdc9995fecd4fb7d266 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:07.220710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:07.221276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:07.221276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:07.247442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:07.250268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:07.336834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1028/2054] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:07.343830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:07.345871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:07.510952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6007-60b1-453f-a00a-70a1e8d2b1d9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:07.522840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3b9f2-0896-4293-ba08-9afd9925db62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:07.526310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3b9f2-0896-4293-ba08-9afd9925db62 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:07.526655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3b9f2-0896-4293-ba08-9afd9925db62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:07.526939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3b9f2-0896-4293-ba08-9afd9925db62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:07.573984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3b9f2-0896-4293-ba08-9afd9925db62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:07.576774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3b9f2-0896-4293-ba08-9afd9925db62 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:07.601463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1027/2055] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:08.595434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:08.597140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:08.617336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036929b9-861f-4b41-825e-39427d77d1ce None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:08.648040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:08.648779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:08.649057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:08.650079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] PATH_INFO: `/v3/projects/57cd54d9d3e7493cb82918e73f8626df` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:08.654323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cd54d9d3e7493cb82918e73f8626df)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:08.658341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:08.746016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cd54d9d3e7493cb82918e73f8626df', 'request_id': 'req-4877615b-003f-40b8-9216-418d1c09faef'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:08.757865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cd54d9d3e7493cb82918e73f8626df', 'request_id': 'req-4877615b-003f-40b8-9216-418d1c09faef'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:08.773360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cd54d9d3e7493cb82918e73f8626df', 'request_id': 'req-4877615b-003f-40b8-9216-418d1c09faef'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:08.791570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] The token cache is being invalidate because project 57cd54d9d3e7493cb82918e73f8626df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:08.792149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:08.793145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4877615b-003f-40b8-9216-418d1c09faef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:08.794949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1029/2056] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:08 2022] DELETE /identity/v3/projects/57cd54d9d3e7493cb82918e73f8626df => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:08.800049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:08.800927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:08.800927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:08.874917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:08.879562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:08.976036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1028/2057] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:08.985581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:08.985982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:08.986339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.014411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:09.016166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:09.031070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1030/2058] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:09.035355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.048311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe079c8e-0e97-4a54-818d-7c53c0001855 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:09.053177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:09.053797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.054077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.054367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.055013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:09.055322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:09.057019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:09.072271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1029/2059] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:17:09.076419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.086953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978ca0a-e282-4d04-b5f6-a5749342f7f2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:09.095706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:09.096506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.097021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.097509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.098345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:09.100553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:09.180488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1031/2060] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:09 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:09.185795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.197907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b6279-5a3e-40cb-9b23-61031c4fe25c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:09.282235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:09.386705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:09.387425 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.387711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.388006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.389240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:09.392127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:09.432240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1030/2061] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:09 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:09.437321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.465245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af941008-de13-4fc1-aeff-7ff6b9e47cd1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:09.465245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:09.465245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.465245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.465715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.465715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:09.465715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:09.497163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1032/2062] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:09.497371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.535468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cfc9-6a2e-4a8f-9fd1-6e10990e6692 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:09.550326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd280a-0b9e-4105-baab-b5b189c8c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:09.553739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd280a-0b9e-4105-baab-b5b189c8c4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.554037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd280a-0b9e-4105-baab-b5b189c8c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.554321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd280a-0b9e-4105-baab-b5b189c8c4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.557035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd280a-0b9e-4105-baab-b5b189c8c4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:09.560221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd280a-0b9e-4105-baab-b5b189c8c4c8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:09.594800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1031/2063] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:09.599446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.635652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0f375-696a-4100-b39b-9aad19abe65b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:09.643020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:09.643876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.644662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.645418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] PATH_INFO: `/v3/projects/224754f7d09c49198bb852f91f340e4a/users/39b50bd816164229b8d4f9f416b40e40/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.673361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224754f7d09c49198bb852f91f340e4a, user_id=39b50bd816164229b8d4f9f416b40e40, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:09.678400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:09.724014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1033/2064] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:09 2022] PUT /identity/v3/projects/224754f7d09c49198bb852f91f340e4a/users/39b50bd816164229b8d4f9f416b40e40/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:09.726553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:09.727109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:09.727545 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:09.762946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] MFA Rules not processed for user `39b50bd816164229b8d4f9f416b40e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:09.764816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:09.929471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1032/2065] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:09.955053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:09.962292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:09.981249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:10.086859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:10.092363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:10.229060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5a1bd-0f18-4903-846e-bec361271090 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:10.241339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbe37e-ead9-4ca9-8968-5a39b3eab4ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:10.244774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbe37e-ead9-4ca9-8968-5a39b3eab4ff service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:10.246232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbe37e-ead9-4ca9-8968-5a39b3eab4ff service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:10.248922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbe37e-ead9-4ca9-8968-5a39b3eab4ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:10.263933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbe37e-ead9-4ca9-8968-5a39b3eab4ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:10.266049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbe37e-ead9-4ca9-8968-5a39b3eab4ff service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:10.278649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490334c1-a561-4be6-a258-941fd2e8addc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:10.281557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1034/2066] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:10.285650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:10.286925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:10.287427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:10.287914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] PATH_INFO: `/v3/users/d1d3dccd729c464cb5d52abfe3d6e2e4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:10.289491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d3dccd729c464cb5d52abfe3d6e2e4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:10.292303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:10.396780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:10.444521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d3dccd729c464cb5d52abfe3d6e2e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:10.445931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d3dccd729c464cb5d52abfe3d6e2e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:10.446776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d3dccd729c464cb5d52abfe3d6e2e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:10.467573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d3dccd729c464cb5d52abfe3d6e2e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:10.491492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d3dccd729c464cb5d52abfe3d6e2e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:10.518959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1033/2067] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:10 2022] DELETE /identity/v3/users/d1d3dccd729c464cb5d52abfe3d6e2e4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:10.523227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:10.523749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:10.524177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:10.555786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:10.558375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:10.644312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1035/2068] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:10.656128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:10.658479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:10.777691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cd289-58e3-4085-9d98-9c37c5b6acab admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:10.783002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:10.784620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:10.785078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:10.785512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:10.818546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:10.820737 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:10.844946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1034/2069] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:11.456458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:11.457883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:11.473760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5a153-afc9-41f7-9ecc-ce672c576ab7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:11.480603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:11.481613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:11.482630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:11.483086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] PATH_INFO: `/v3/projects/0e0a690be6cd4c28aa9f38edb76bee0c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:11.485218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0a690be6cd4c28aa9f38edb76bee0c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:11.487534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:11.555928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0a690be6cd4c28aa9f38edb76bee0c', 'request_id': 'req-dae2154e-eb27-458c-8038-2b89a8656a09'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.564548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0a690be6cd4c28aa9f38edb76bee0c', 'request_id': 'req-dae2154e-eb27-458c-8038-2b89a8656a09'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.574578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0a690be6cd4c28aa9f38edb76bee0c', 'request_id': 'req-dae2154e-eb27-458c-8038-2b89a8656a09'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.588451 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] The token cache is being invalidate because project 0e0a690be6cd4c28aa9f38edb76bee0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:11.588736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.590324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.605426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1036/2070] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:11 2022] DELETE /identity/v3/projects/0e0a690be6cd4c28aa9f38edb76bee0c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:11.610880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:11.631987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef003aa-c41a-4ceb-b13c-21ed7ea12697 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:11.786401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:11.787057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:11.787342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:11.787618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] PATH_INFO: `/v3/users/db2b3597e34845bcad4b94c96505691c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:11.788845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2b3597e34845bcad4b94c96505691c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:11.790894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:11.869300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2b3597e34845bcad4b94c96505691c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.869964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2b3597e34845bcad4b94c96505691c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.870577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2b3597e34845bcad4b94c96505691c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.880950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:11.888773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2b3597e34845bcad4b94c96505691c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.915535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-02e83cd5-a2da-4d63-a139-26f81b342357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2b3597e34845bcad4b94c96505691c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:11.932847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1035/2071] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:11 2022] DELETE /identity/v3/users/db2b3597e34845bcad4b94c96505691c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:12.547073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:12.548582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:12.574786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2154e-eb27-458c-8038-2b89a8656a09 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:12.590647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:12.679103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:12.679751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:12.680029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:12.680336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] PATH_INFO: `/v3/projects/d1c75e3109544cef8b373cbf0ef2b698` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:12.681662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c75e3109544cef8b373cbf0ef2b698)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:12.684224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:12.759341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c75e3109544cef8b373cbf0ef2b698', 'request_id': 'req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:12.771325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c75e3109544cef8b373cbf0ef2b698', 'request_id': 'req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:12.781089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c75e3109544cef8b373cbf0ef2b698', 'request_id': 'req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:12.839849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] The token cache is being invalidate because project d1c75e3109544cef8b373cbf0ef2b698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:12.840178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:12.841088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f6972375-8a00-48c2-8d9b-bbadd1d11b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:12.847880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1037/2072] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:12 2022] DELETE /identity/v3/projects/d1c75e3109544cef8b373cbf0ef2b698 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:12.852135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:12.852680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:12.852874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:12.886950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:12.909339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:13.027433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1036/2073] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:13.028749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.029483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.029994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.054691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:13.056259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:13.066339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1038/2074] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:13.070457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.082771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae102d4-327e-4b62-8503-ad9dfda74d2c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.088082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.088804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.089090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.089430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.090241 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:13.090583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.092616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.106635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1037/2075] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:13.111566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.125562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d6f2e-8841-4c0a-ad99-5234f823dc44 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.131968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.132995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.133444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.133921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.134893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.138935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.183852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1039/2076] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:13 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:13.190292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.205911 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0daa-3714-4c85-b909-8a298339e8a4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.290906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.292010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.292573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.293061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.294147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.300844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.344987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1038/2077] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:13 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:13.349163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.360551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd13c6-20f0-43a8-a79f-387f20ba5f83 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.370985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.371808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.371903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.372181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.373139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.374751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.392133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1040/2078] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:13.396338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.409671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baedfa-bb19-43ec-99bb-05c3c720df35 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.414767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e27a2-6dc5-40c8-9064-ca65dbe74d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.415521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e27a2-6dc5-40c8-9064-ca65dbe74d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.416123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e27a2-6dc5-40c8-9064-ca65dbe74d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.416603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e27a2-6dc5-40c8-9064-ca65dbe74d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.417495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e27a2-6dc5-40c8-9064-ca65dbe74d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.419326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e27a2-6dc5-40c8-9064-ca65dbe74d41 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.434785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1039/2079] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:13.446003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.464319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b2476-7cca-4ce4-b797-d0d4eb380506 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.470677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.472000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.472467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.472826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] PATH_INFO: `/v3/projects/bee48e665fb244869f78a93e091b758b/users/b1743984cf494fe391e55c8cd4699f59/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.505836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee48e665fb244869f78a93e091b758b, user_id=b1743984cf494fe391e55c8cd4699f59, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.508678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.531740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1041/2080] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:13 2022] PUT /identity/v3/projects/bee48e665fb244869f78a93e091b758b/users/b1743984cf494fe391e55c8cd4699f59/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:13.543566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.544224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.544748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.588843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] MFA Rules not processed for user `b1743984cf494fe391e55c8cd4699f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:13.591181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:13.647574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:13.720518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1040/2081] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:13.721800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.787204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:13.815617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958864e-27f8-44f7-900d-f002d47d794e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:13.821898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:13.822532 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:13.822817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:13.823094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:13.823730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:13.825880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:13.875696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1042/2082] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:13 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:13.879763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:13.899082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0815d5-bc1b-42e0-bbbe-e5406f6ad429 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:14.032519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:14.033567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.034162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.034636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.035688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:14.039024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:14.078086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1041/2083] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:13 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:14.078318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:14.096533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a315b34-a365-426b-bc34-241fa612214a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:14.102822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:14.103807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.104329 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.104815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.105681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:14.108511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:14.133239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:14.143500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1043/2084] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:14.147695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc75ca-1721-4559-8162-35d399570c45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:14.155015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:14.155811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.156159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.156547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] PATH_INFO: `/v3/projects/76cfb52b6edc45e3ac2b0ab06ccb2f1a/users/cbccee0edc2a4b1b86158f36773e961f/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.189843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76cfb52b6edc45e3ac2b0ab06ccb2f1a, user_id=cbccee0edc2a4b1b86158f36773e961f, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:14.193251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:14.218513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1042/2085] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:14 2022] PUT /identity/v3/projects/76cfb52b6edc45e3ac2b0ab06ccb2f1a/users/cbccee0edc2a4b1b86158f36773e961f/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:14.223031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:14.237749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6c36a-8599-49b4-91b3-d412b90e0bbd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:14.342880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53c094-8372-4bc9-863b-b89ed9622518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:14.343653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53c094-8372-4bc9-863b-b89ed9622518 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.343986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53c094-8372-4bc9-863b-b89ed9622518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.344340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53c094-8372-4bc9-863b-b89ed9622518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.345039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53c094-8372-4bc9-863b-b89ed9622518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:14.347522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53c094-8372-4bc9-863b-b89ed9622518 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:14.372043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1044/2086] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:14.373442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:14.394342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae018265-2a71-4e77-b93c-c9934846b802 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:14.407985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:14.407985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.407985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.408595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] PATH_INFO: `/v3/projects/76cfb52b6edc45e3ac2b0ab06ccb2f1a/users/cbccee0edc2a4b1b86158f36773e961f/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.410695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76cfb52b6edc45e3ac2b0ab06ccb2f1a, user_id=cbccee0edc2a4b1b86158f36773e961f, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:14.413848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:14.438919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dce23-2d9a-4223-a447-e35ef6b11b49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.439319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dce23-2d9a-4223-a447-e35ef6b11b49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.439892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dce23-2d9a-4223-a447-e35ef6b11b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.443251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1043/2087] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:14 2022] PUT /identity/v3/projects/76cfb52b6edc45e3ac2b0ab06ccb2f1a/users/cbccee0edc2a4b1b86158f36773e961f/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:14.510075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c21dce23-2d9a-4223-a447-e35ef6b11b49 None None] MFA Rules not processed for user `cbccee0edc2a4b1b86158f36773e961f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:14.512084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c21dce23-2d9a-4223-a447-e35ef6b11b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:14.620784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1045/2088] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:14.633733 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:14.636321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:14.758517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219caa20-a352-4fda-9499-050aa96d095f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:14.763742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:14.764406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:14.764689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:14.765445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:14.813796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:14.862342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:14.866038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:14.885464 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1044/2089] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:18.919724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:18.920611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:18.920611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:18.964776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] MFA Rules not processed for user `7ea15f54d31149ea948650cdd363d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:18.966132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:19.007671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:19.120750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1046/2090] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:19.127516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:19.128983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:19.237162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223376-1562-4a86-8740-02e85572c28c service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:19.245020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:19.245657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:19.245938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:19.246205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:19.247281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:19.249699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:19.293849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1045/2091] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:19 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:19.299544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:19.312456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb01704-50ce-47a6-a4d2-ec024ca6acd1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:19.408865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:19.409981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:19.409981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:19.410208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:19.410704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:19.412700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:19.451871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1047/2092] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:19 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:19.455989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:19.487942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b50e5d-8226-416f-b151-e91041b7c903 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:19.496970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:19.496970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:19.497326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:19.498248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:19.500014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:19.502292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:19.517649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1046/2093] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:19.522078 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:19.536707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71196e4-91e1-4141-8db5-af4000bd7cac admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:19.554644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:19.555397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:19.555670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:19.555942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] PATH_INFO: `/v3/projects/f4fdcdd025f84466b5c5c9fd4ac5234d/users/3630f86adf69408396086978eda5ee58/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:19.593435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fdcdd025f84466b5c5c9fd4ac5234d, user_id=3630f86adf69408396086978eda5ee58, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:19.601960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:19.641594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:19.642432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1048/2094] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:19 2022] PUT /identity/v3/projects/f4fdcdd025f84466b5c5c9fd4ac5234d/users/3630f86adf69408396086978eda5ee58/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:19.656569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ccccf-c45f-462a-89d3-d70d356a8e49 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:19.822230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:19.822577 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:19.822862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:19.826788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:19.827609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:19.829531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:19.847461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:19.850035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1047/2095] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:19.866140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754fc4d-258d-4a77-9aa0-66816dd36ee1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:19.880097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:19.880821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:19.881206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:19.881497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] PATH_INFO: `/v3/projects/f4fdcdd025f84466b5c5c9fd4ac5234d/users/3630f86adf69408396086978eda5ee58/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:19.894633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fdcdd025f84466b5c5c9fd4ac5234d, user_id=3630f86adf69408396086978eda5ee58, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:19.901824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:19.916503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1049/2096] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:19 2022] PUT /identity/v3/projects/f4fdcdd025f84466b5c5c9fd4ac5234d/users/3630f86adf69408396086978eda5ee58/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:22.289895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:22.372058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:22.377704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24686095-9c3d-4b7c-9899-77f790aa86c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:22.504094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:22.505029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:22.505385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:22.505751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:22.662579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:22.664625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:22.683722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1048/2097] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:23.711922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:23.714597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:23.802164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439a7de-34f2-4cab-a526-ffd64a70b9bc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:23.806983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:23.807621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:23.807899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:23.808168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] PATH_INFO: `/v3/users/c6f76402e6574466b0df7ca1dde62377` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:23.809336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f76402e6574466b0df7ca1dde62377)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:23.811299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:23.828416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:23.863635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f76402e6574466b0df7ca1dde62377'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:23.878045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f76402e6574466b0df7ca1dde62377'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:23.895898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f76402e6574466b0df7ca1dde62377'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:23.896569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f76402e6574466b0df7ca1dde62377'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:23.897192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f76402e6574466b0df7ca1dde62377'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:23.912295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1050/2098] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:23 2022] DELETE /identity/v3/users/c6f76402e6574466b0df7ca1dde62377 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:24.258500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:24.286053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce7c3e-55a6-48cd-a9a2-af565d7d6607 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:24.469743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:24.471007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:24.471289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:24.471559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] PATH_INFO: `/v3/projects/46ab441b7dcd4611a77ed792c4bc8a9f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:24.472942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ab441b7dcd4611a77ed792c4bc8a9f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:24.476982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:24.540105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ab441b7dcd4611a77ed792c4bc8a9f', 'request_id': 'req-59710952-9e5f-4f58-8fb4-f80b08220e58'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.545853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:24.551914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ab441b7dcd4611a77ed792c4bc8a9f', 'request_id': 'req-59710952-9e5f-4f58-8fb4-f80b08220e58'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.562101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ab441b7dcd4611a77ed792c4bc8a9f', 'request_id': 'req-59710952-9e5f-4f58-8fb4-f80b08220e58'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.584540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] The token cache is being invalidate because project 46ab441b7dcd4611a77ed792c4bc8a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:24.584983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.585752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.588450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1049/2099] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:24 2022] DELETE /identity/v3/projects/46ab441b7dcd4611a77ed792c4bc8a9f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:24.591508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:24.593697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:24.611403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a53e7-daa2-4a17-b4f3-2bb97dfe8de2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:24.714273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:24.729042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:24.729276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:24.729521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] PATH_INFO: `/v3/users/ca5dbf32e4f946e794bf90469651f78f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:24.737098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5dbf32e4f946e794bf90469651f78f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:24.739139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:24.842492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5dbf32e4f946e794bf90469651f78f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.869037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5dbf32e4f946e794bf90469651f78f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.892157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5dbf32e4f946e794bf90469651f78f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.893516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5dbf32e4f946e794bf90469651f78f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.894777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-42deaaa3-7c96-4741-8ca1-1d8a8b0696d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5dbf32e4f946e794bf90469651f78f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:24.932835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1051/2100] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:24 2022] DELETE /identity/v3/users/ca5dbf32e4f946e794bf90469651f78f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:25.450329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:25.453319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:25.470852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59710952-9e5f-4f58-8fb4-f80b08220e58 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:25.570572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:25.571742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:25.572246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:25.572765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] PATH_INFO: `/v3/projects/992c975ce7194088837e4f12aba0c0dd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:25.574864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=992c975ce7194088837e4f12aba0c0dd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:25.578123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:25.652079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '992c975ce7194088837e4f12aba0c0dd', 'request_id': 'req-4f4479ab-2359-46ba-98fe-ec2ff6284b12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:25.663584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '992c975ce7194088837e4f12aba0c0dd', 'request_id': 'req-4f4479ab-2359-46ba-98fe-ec2ff6284b12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:25.672971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '992c975ce7194088837e4f12aba0c0dd', 'request_id': 'req-4f4479ab-2359-46ba-98fe-ec2ff6284b12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:25.698353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] The token cache is being invalidate because project 992c975ce7194088837e4f12aba0c0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:25.698653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:25.699548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f4479ab-2359-46ba-98fe-ec2ff6284b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:25.702833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1050/2101] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:25 2022] DELETE /identity/v3/projects/992c975ce7194088837e4f12aba0c0dd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:25.707767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:25.708060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:25.708367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:25.729053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:25.730585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:25.746144 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:25.826662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1052/2102] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:25.835877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:25.836182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:25.836481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:25.897800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:25.901388 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:25.928509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1051/2103] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:25.932634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:25.954287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67664c9d-fba5-4441-8366-7e827a2f9950 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:25.978360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:25.979729 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:25.980018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:25.980322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:25.980993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:25.981534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:25.984373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.000030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1053/2104] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:17:26.005185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.017746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375f56e-8aaa-4a91-9b3e-50fbadc4bd8d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.026153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.026894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.027248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.027481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.028262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.031597 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.085753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1052/2105] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:26 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:26.093423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.117669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c4ad0-2669-43bd-a5f2-494b9360323e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.272983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.274921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.277071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.281182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.282538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.285989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.338435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1054/2106] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:26 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:26.342646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.359899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81140de7-b7f7-4dd1-87f7-216073c301c0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.365584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.366396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.366775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.367311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.368095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.370097 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.386099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1053/2107] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:26.390607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.403009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3780ca-8998-469e-98db-0e73c2100025 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.408528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c92b9-6ee3-437d-b0df-94071c2fbc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.409165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c92b9-6ee3-437d-b0df-94071c2fbc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.409448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c92b9-6ee3-437d-b0df-94071c2fbc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.409871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c92b9-6ee3-437d-b0df-94071c2fbc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.410479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c92b9-6ee3-437d-b0df-94071c2fbc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.412370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c92b9-6ee3-437d-b0df-94071c2fbc09 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.425998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1055/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:26.430697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.447974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401469f-6982-43ba-8881-56ff51418abd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.454459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.455843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.456363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.456494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] PATH_INFO: `/v3/projects/80352f59b9184b0986d73586d753ed27/users/2ce89fc1926a44629a331184befa4f05/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.477140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80352f59b9184b0986d73586d753ed27, user_id=2ce89fc1926a44629a331184befa4f05, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.480031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.501234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1054/2109] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:26 2022] PUT /identity/v3/projects/80352f59b9184b0986d73586d753ed27/users/2ce89fc1926a44629a331184befa4f05/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:26.511987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.512320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.512575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.569175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] MFA Rules not processed for user `2ce89fc1926a44629a331184befa4f05`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:26.570349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:26.681970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1056/2110] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:26.687100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.702457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:26.777877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9164-fece-42a1-b28a-9777a18629e9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.785568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.785568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.786128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.786468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.787252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.789772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:26.824932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1055/2111] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:26 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:26.829886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:26.847018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e328e5d-7fd9-469e-87a4-9cd3c3c449b7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:26.854972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:26.982039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:26.982809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:26.983701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:26.984105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:26.984886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:26.989961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.052866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1057/2112] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:26 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:27.057520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:27.074970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9b1f-3fe6-4da2-be79-493cee58b2ad admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:27.086325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bf89c-9d4c-4232-a1fd-e4d9a5b0f1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:27.086974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf89c-9d4c-4232-a1fd-e4d9a5b0f1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.087269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf89c-9d4c-4232-a1fd-e4d9a5b0f1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.087526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf89c-9d4c-4232-a1fd-e4d9a5b0f1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.088217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bf89c-9d4c-4232-a1fd-e4d9a5b0f1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:27.090875 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bf89c-9d4c-4232-a1fd-e4d9a5b0f1d2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.109983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1056/2113] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:27.114501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:27.125497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eca42-2957-46f8-86d7-6a99979ab223 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:27.131301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:27.132013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.132337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.132609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] PATH_INFO: `/v3/projects/f37236a9facf4fb5a3f73500b01851aa/users/8f9f9a99aa794850bdfe2d5861fcafb4/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.151203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37236a9facf4fb5a3f73500b01851aa, user_id=8f9f9a99aa794850bdfe2d5861fcafb4, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:27.155303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.175708 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1058/2114] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:27 2022] PUT /identity/v3/projects/f37236a9facf4fb5a3f73500b01851aa/users/8f9f9a99aa794850bdfe2d5861fcafb4/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:27.179500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.179993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.180422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.184234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:27.228636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] MFA Rules not processed for user `3630f86adf69408396086978eda5ee58`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:27.230439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:27.310399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836033-1942-480b-bc5e-b07b75684b1a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:27.320885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:27.322731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.323033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.323360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.324036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:27.326742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.342422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1059/2115] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:27.346463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:27.359671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948166e-78b7-45ff-80de-6cd00fe77a97 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:27.365096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa08e2-e9b8-485f-b558-bd4d6bb820b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:27.365771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa08e2-e9b8-485f-b558-bd4d6bb820b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.366106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa08e2-e9b8-485f-b558-bd4d6bb820b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.366376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa08e2-e9b8-485f-b558-bd4d6bb820b4 admin admin] PATH_INFO: `/v3/projects/f37236a9facf4fb5a3f73500b01851aa/users/8f9f9a99aa794850bdfe2d5861fcafb4/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.373513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa08e2-e9b8-485f-b558-bd4d6bb820b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37236a9facf4fb5a3f73500b01851aa, user_id=8f9f9a99aa794850bdfe2d5861fcafb4, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:27.376599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa08e2-e9b8-485f-b558-bd4d6bb820b4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.412966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1060/2116] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:27 2022] PUT /identity/v3/projects/f37236a9facf4fb5a3f73500b01851aa/users/8f9f9a99aa794850bdfe2d5861fcafb4/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:27.414589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.414882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.415159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.437194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1057/2117] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:27.451298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:27.453706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:27.474128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] MFA Rules not processed for user `8f9f9a99aa794850bdfe2d5861fcafb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:27.476790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:27.539227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:27.568224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1061/2118] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:27.571908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcaf2d-2a20-45ed-ba53-93d07b98b27a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:27.575958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:27.580145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:27.581977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:27.646226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315090ad-b1a8-49cc-808d-de5ade9dc63a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:27.649979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:27.650645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.650930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.651202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.654892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:27.655515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:27.655792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:27.656060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:27.657883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:27.720678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:27.722327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.722961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:27.724593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e17c14-c17e-43db-8a71-62b716ac0d04 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:27.734218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1062/2119] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:27.736172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1058/2120] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:28.579658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:28.581127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:28.664880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c405291-6e61-4e68-bd9b-5b3604ed5136 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:28.670170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:28.670881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:28.671223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:28.671734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] PATH_INFO: `/v3/users/b1743984cf494fe391e55c8cd4699f59` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:28.673223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1743984cf494fe391e55c8cd4699f59)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:28.675147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:28.726549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:28.733014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1743984cf494fe391e55c8cd4699f59'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:28.757849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1743984cf494fe391e55c8cd4699f59'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:28.776506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1743984cf494fe391e55c8cd4699f59'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:28.777713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1743984cf494fe391e55c8cd4699f59'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:28.778000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1743984cf494fe391e55c8cd4699f59'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:28.795640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1063/2121] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:28 2022] DELETE /identity/v3/users/b1743984cf494fe391e55c8cd4699f59 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:28.800000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:28.800471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:28.800715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:28.822070 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:28.823590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:28.894057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1059/2122] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:28.907595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:28.909468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:28.994587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8386-02e0-496d-80a2-b0b971e627b0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:28.999020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:28.999730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:29.005742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:29.005742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:29.012664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:29.015035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:29.026663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1064/2123] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:29.580131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:29.581744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:29.622098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627e586-3454-49a5-9be4-a6eeb55cfdbb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:29.637556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:29.639927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:29.643717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:29.644232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] PATH_INFO: `/v3/projects/bee48e665fb244869f78a93e091b758b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:29.650850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee48e665fb244869f78a93e091b758b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:29.655382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:29.781385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:29.795763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee48e665fb244869f78a93e091b758b', 'request_id': 'req-45ac667c-f8e8-4377-954f-b3b39133ba71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:29.810873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee48e665fb244869f78a93e091b758b', 'request_id': 'req-45ac667c-f8e8-4377-954f-b3b39133ba71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:29.845223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee48e665fb244869f78a93e091b758b', 'request_id': 'req-45ac667c-f8e8-4377-954f-b3b39133ba71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:29.875611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] The token cache is being invalidate because project bee48e665fb244869f78a93e091b758b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:29.876390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:29.877355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:29.879186 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1060/2124] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:29 2022] DELETE /identity/v3/projects/bee48e665fb244869f78a93e091b758b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:29.883464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:29.899648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21781b42-c413-488f-85cd-61bb99c013eb service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:30.006621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:30.007289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.007522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.007811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] PATH_INFO: `/v3/users/cbccee0edc2a4b1b86158f36773e961f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.009168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbccee0edc2a4b1b86158f36773e961f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:30.011660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:30.132064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbccee0edc2a4b1b86158f36773e961f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.204142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:30.206987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbccee0edc2a4b1b86158f36773e961f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.215663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbccee0edc2a4b1b86158f36773e961f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.217246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbccee0edc2a4b1b86158f36773e961f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.219299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbccee0edc2a4b1b86158f36773e961f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.233270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1065/2125] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:29 2022] DELETE /identity/v3/users/cbccee0edc2a4b1b86158f36773e961f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:30.573111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:30.574640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:30.598128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac667c-f8e8-4377-954f-b3b39133ba71 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:30.672924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:30.673561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.673837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.674106 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] PATH_INFO: `/v3/projects/76cfb52b6edc45e3ac2b0ab06ccb2f1a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.675226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76cfb52b6edc45e3ac2b0ab06ccb2f1a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:30.677171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:30.736139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76cfb52b6edc45e3ac2b0ab06ccb2f1a', 'request_id': 'req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.739222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:30.741269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:30.746163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76cfb52b6edc45e3ac2b0ab06ccb2f1a', 'request_id': 'req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.754309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76cfb52b6edc45e3ac2b0ab06ccb2f1a', 'request_id': 'req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.756950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8b27-05ab-44e0-889a-d55da700439b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:30.768341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] The token cache is being invalidate because project 76cfb52b6edc45e3ac2b0ab06ccb2f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:30.768952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.770007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8aace4cf-a298-4a34-91a4-9a55ed0b4f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.771637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1061/2126] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:30 2022] DELETE /identity/v3/projects/76cfb52b6edc45e3ac2b0ab06ccb2f1a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:30.777789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850905b-58fa-413c-9e51-e15d6f11329f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.778195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850905b-58fa-413c-9e51-e15d6f11329f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.778705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850905b-58fa-413c-9e51-e15d6f11329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.803058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d850905b-58fa-413c-9e51-e15d6f11329f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:30.804184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d850905b-58fa-413c-9e51-e15d6f11329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:30.822138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:30.822771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.823137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.823521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] PATH_INFO: `/v3/users/2ce89fc1926a44629a331184befa4f05` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.826932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce89fc1926a44629a331184befa4f05)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:30.828876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:30.867568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:30.894900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1062/2127] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:30.895751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce89fc1926a44629a331184befa4f05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.896498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.896970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.897238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.923562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce89fc1926a44629a331184befa4f05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.926157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:30.927547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:30.933763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce89fc1926a44629a331184befa4f05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.935390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce89fc1926a44629a331184befa4f05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.936304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4c7836b-5839-46be-a342-9109ff377fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce89fc1926a44629a331184befa4f05'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:30.937278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1063/2128] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:30.941926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:30.952890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6283600-5331-482c-8b19-cb65c17874f5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:30.957942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:30.958669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.958990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1066/2129] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:30 2022] DELETE /identity/v3/users/2ce89fc1926a44629a331184befa4f05 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:30.961959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.961959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.961959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:30.961959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:30.962695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:30.963669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:30.964006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:30.964301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:30.983447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1064/2130] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:17:30.988290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:30.990351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:30.990525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:30.991759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:31.006411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b15567-fa16-40a1-8645-e6925b9fe4d7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.084431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:31.105399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1067/2131] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:31.116811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.120367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:31.161408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.162041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.162318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.162591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.163224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.165228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.230704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1065/2132] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:30 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:31.238063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.245560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7016f9-6a92-441a-a47f-050a60b0a170 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.347629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.348290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.348574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.349111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.360414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ff476-d4a3-4573-929b-56c09e40137c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.362991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.365349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.366232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.367119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.367627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.368093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.371100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.375706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.387589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1068/2133] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:31.412360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1066/2134] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:31 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:31.413710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.455635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3b851-cba8-4b57-97f7-02e1fdbea8e5 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.466197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.468533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.469039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.469631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.470570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.473581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.491239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1069/2135] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:31.509582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.523103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbf887-9144-4d18-88c7-4d9befb31090 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.530943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed01eb-6362-4cf0-94ce-7d6b56957c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.537056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed01eb-6362-4cf0-94ce-7d6b56957c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.537486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed01eb-6362-4cf0-94ce-7d6b56957c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.537770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed01eb-6362-4cf0-94ce-7d6b56957c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.538845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed01eb-6362-4cf0-94ce-7d6b56957c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.540488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed01eb-6362-4cf0-94ce-7d6b56957c03 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.555762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1067/2136] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:31.564800 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.579644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1801ee-e0c4-425c-9223-3dd816ff03f2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.586256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.586990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.587280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.587555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] PATH_INFO: `/v3/projects/62761b2573e1490fb2583434b0d50ff1/users/b555252579604e6c9a0a403f9d93537f/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.617056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62761b2573e1490fb2583434b0d50ff1, user_id=b555252579604e6c9a0a403f9d93537f, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.623500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.644410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1070/2137] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:31 2022] PUT /identity/v3/projects/62761b2573e1490fb2583434b0d50ff1/users/b555252579604e6c9a0a403f9d93537f/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:31.656975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.657274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.657662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.691810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] MFA Rules not processed for user `b555252579604e6c9a0a403f9d93537f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:31.693038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:31.725200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:31.755699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.757801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:31.786919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1068/2138] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:31.798936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.800764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:31.853025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ed075-96d0-4c86-b0e8-bc955bfcb9e1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.858538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.859253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.859539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.859815 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] PATH_INFO: `/v3/projects/80352f59b9184b0986d73586d753ed27` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.860990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80352f59b9184b0986d73586d753ed27)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.862915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.863128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fa1ed-1bca-495f-875f-f88b400874f1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.866919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.867519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.867849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.868628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.881463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.882919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:31.892882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1069/2139] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:31.895280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:31.913134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.924420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8e2f-b09f-4a61-98b2-3ff707f691d7 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:31.927447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80352f59b9184b0986d73586d753ed27', 'request_id': 'req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:31.933732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80352f59b9184b0986d73586d753ed27', 'request_id': 'req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:31.944858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80352f59b9184b0986d73586d753ed27', 'request_id': 'req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:31.957282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] The token cache is being invalidate because project 80352f59b9184b0986d73586d753ed27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:31.957837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:31.958530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:31.963045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1071/2140] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:31 2022] DELETE /identity/v3/projects/80352f59b9184b0986d73586d753ed27 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:31.964276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:31.965721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:31.992660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:31.993292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:31.993575 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:31.993855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] PATH_INFO: `/v3/users/b555252579604e6c9a0a403f9d93537f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:31.995012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b555252579604e6c9a0a403f9d93537f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:31.996991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:32.045159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795c7ea-ff8b-4f70-8a8b-ba3d4c5ae873 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:32.052754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:32.053557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:32.053708 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:32.053939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] PATH_INFO: `/v3/users/8f9f9a99aa794850bdfe2d5861fcafb4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:32.055838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f9f9a99aa794850bdfe2d5861fcafb4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:32.058032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:32.059285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b555252579604e6c9a0a403f9d93537f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.060798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b555252579604e6c9a0a403f9d93537f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.061490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b555252579604e6c9a0a403f9d93537f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.076977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b555252579604e6c9a0a403f9d93537f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.088116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df51aa28-4212-4c9a-be3d-02c3b38611eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b555252579604e6c9a0a403f9d93537f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.095122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:32.108479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1070/2141] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:31 2022] DELETE /identity/v3/users/b555252579604e6c9a0a403f9d93537f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:32.112234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:32.112566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:32.113021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:32.143917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:32.144165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f9f9a99aa794850bdfe2d5861fcafb4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.145745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:32.162334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f9f9a99aa794850bdfe2d5861fcafb4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.172687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f9f9a99aa794850bdfe2d5861fcafb4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.174377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f9f9a99aa794850bdfe2d5861fcafb4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.175096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f9f9a99aa794850bdfe2d5861fcafb4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:32.198807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1072/2142] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:31 2022] DELETE /identity/v3/users/8f9f9a99aa794850bdfe2d5861fcafb4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:32.217603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1071/2143] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:32.227805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:32.229380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:32.293566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:32.338743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009630b-a85e-4a01-a9ea-3dfda7683b23 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:32.343988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe1114-39b6-49b9-98bd-0b49f1fa7117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:32.344741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1114-39b6-49b9-98bd-0b49f1fa7117 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:32.345019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1114-39b6-49b9-98bd-0b49f1fa7117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:32.345312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1114-39b6-49b9-98bd-0b49f1fa7117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:32.424964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1114-39b6-49b9-98bd-0b49f1fa7117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:32.426436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1114-39b6-49b9-98bd-0b49f1fa7117 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:32.441040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1073/2144] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:32.845485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:32.847232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:32.871530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aa934-371a-4a75-b662-56bc9f9e470f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:32.883052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:32.884098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:32.889822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:32.890207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] PATH_INFO: `/v3/projects/f37236a9facf4fb5a3f73500b01851aa` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:32.893044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37236a9facf4fb5a3f73500b01851aa)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:32.895077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:32.996945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37236a9facf4fb5a3f73500b01851aa', 'request_id': 'req-e10cdca5-1e87-4256-b522-4ee385ec1a31'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.020825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37236a9facf4fb5a3f73500b01851aa', 'request_id': 'req-e10cdca5-1e87-4256-b522-4ee385ec1a31'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.041929 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37236a9facf4fb5a3f73500b01851aa', 'request_id': 'req-e10cdca5-1e87-4256-b522-4ee385ec1a31'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.057820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] The token cache is being invalidate because project f37236a9facf4fb5a3f73500b01851aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:33.058432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.059762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.062633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1072/2145] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:32 2022] DELETE /identity/v3/projects/f37236a9facf4fb5a3f73500b01851aa => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:33.069162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969fa78-f35e-4f9f-a442-c69c404aa698 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.069768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969fa78-f35e-4f9f-a442-c69c404aa698 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.069871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969fa78-f35e-4f9f-a442-c69c404aa698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.097002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e969fa78-f35e-4f9f-a442-c69c404aa698 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:33.097002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e969fa78-f35e-4f9f-a442-c69c404aa698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:33.098566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.100048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:33.215781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1074/2146] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:33.220423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdca5-1e87-4256-b522-4ee385ec1a31 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.224571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f191579-d85a-4740-8876-b773b306061b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.224880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f191579-d85a-4740-8876-b773b306061b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.225154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f191579-d85a-4740-8876-b773b306061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.226117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.226838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.227183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.227524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] PATH_INFO: `/v3/projects/62761b2573e1490fb2583434b0d50ff1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.233686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62761b2573e1490fb2583434b0d50ff1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.238853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.265281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1f191579-d85a-4740-8876-b773b306061b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:33.266722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f191579-d85a-4740-8876-b773b306061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:33.271838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:33.282448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1075/2147] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:33.287243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f191579-d85a-4740-8876-b773b306061b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.299158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f191579-d85a-4740-8876-b773b306061b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.327529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62761b2573e1490fb2583434b0d50ff1', 'request_id': 'req-ff445822-2400-4155-a50f-690ac62e6316'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.339296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62761b2573e1490fb2583434b0d50ff1', 'request_id': 'req-ff445822-2400-4155-a50f-690ac62e6316'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.348296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62761b2573e1490fb2583434b0d50ff1', 'request_id': 'req-ff445822-2400-4155-a50f-690ac62e6316'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.367432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] The token cache is being invalidate because project 62761b2573e1490fb2583434b0d50ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:33.368235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.369311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ff445822-2400-4155-a50f-690ac62e6316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:33.372242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1073/2148] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:33 2022] DELETE /identity/v3/projects/62761b2573e1490fb2583434b0d50ff1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:33.379664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.380044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.380828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.415480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.415747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:33.416821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.417089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:33.417468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.417943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.418811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:33.420028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.422159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c6a3d-de77-48c6-9d04-9872cb93f04a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.436401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1074/2149] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:33.439731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1076/2150] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:33.444908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.446538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:33.456010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.456698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.456977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.473065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b4c1-f086-4e3b-9268-d605d4b5025f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.478466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.479163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.479447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.479985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.481072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.481482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:33.483987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:33.487082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.495982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1077/2151] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:33.501019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.514923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:33.524354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6bd07-c01e-45e3-9caf-afdf6c316514 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.525449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1075/2152] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:33.530259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.656115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e19ccb-425d-4c76-b55a-abbe71f89974 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.680270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.680761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.681404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.681685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.681954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.682649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:33.682926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.684832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.685647 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.686017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.686946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.688779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.691459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.707299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1078/2153] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:33.712725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.730824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d13bb-4ec0-4d9f-8b03-6ef2d5ffbf32 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.744877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.746535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1076/2154] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:33.751383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.751768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.752062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.752317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.753171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.755498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.769324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4660f68-338c-4134-bf4d-07d3ad17e5b4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.777695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.779355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.781038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.781038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.782546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.783265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.800254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1077/2155] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:33.807000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.814504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1079/2156] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:33.818739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.834832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14cb77-4e7c-471e-9709-685b7cd93307 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.927784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:33.948536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.949921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.949921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.949921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.951364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.952457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:33.974081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37945cd-ede3-48b4-8c76-39a5da33f80a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:33.986514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:33.986694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1078/2157] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:33.987672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:33.988711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:33.989403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:33.989905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:33.990960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:33.994063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.009354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7c5f6-cb83-47a7-aaed-3a7c902e4bda admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.030674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6980a-91ce-4a99-80e5-7ac93a94f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.031571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6980a-91ce-4a99-80e5-7ac93a94f052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.031862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6980a-91ce-4a99-80e5-7ac93a94f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.032142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6980a-91ce-4a99-80e5-7ac93a94f052 admin admin] PATH_INFO: `/v3/projects/be2ee6fbdcd147e8baecb701df0849d4/users/00d2979da0f84f359fee4207c7a0b694/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.047818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1080/2158] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:33 2022] POST /identity/v3/users => generated 280 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:34.052752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.067084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6980a-91ce-4a99-80e5-7ac93a94f052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2ee6fbdcd147e8baecb701df0849d4, user_id=00d2979da0f84f359fee4207c7a0b694, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.086109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6980a-91ce-4a99-80e5-7ac93a94f052 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.086315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bffe3-4c93-42cb-b28d-235503e55420 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.096267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.096267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.096267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.096267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.096882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.102659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.122846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1081/2159] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:34.122960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1079/2160] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:33 2022] PUT /identity/v3/projects/be2ee6fbdcd147e8baecb701df0849d4/users/00d2979da0f84f359fee4207c7a0b694/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:34.125897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.135729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.136463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.136923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.140356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bba46-a3fb-42a5-b1df-5790842bf2dc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.201541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] MFA Rules not processed for user `00d2979da0f84f359fee4207c7a0b694`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:34.204379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:34.249777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:34.277094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.277714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.278081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.278349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.279772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.281506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.296253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1082/2161] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:34.298629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.313523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866a98e-e27e-41fd-8bd1-33a6a50ce935 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.322824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497a153-4be4-4ac1-a0cb-848a57071c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.323498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497a153-4be4-4ac1-a0cb-848a57071c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.323771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497a153-4be4-4ac1-a0cb-848a57071c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.324067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497a153-4be4-4ac1-a0cb-848a57071c88 admin admin] PATH_INFO: `/v3/projects/2706e2f2b8cf490f8cc2142ff5f59293/users/0c3a70ba657a419990a55c448087859a/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.350830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497a153-4be4-4ac1-a0cb-848a57071c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2706e2f2b8cf490f8cc2142ff5f59293, user_id=0c3a70ba657a419990a55c448087859a, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.355448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497a153-4be4-4ac1-a0cb-848a57071c88 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.380833 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1083/2162] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:34 2022] PUT /identity/v3/projects/2706e2f2b8cf490f8cc2142ff5f59293/users/0c3a70ba657a419990a55c448087859a/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:34.395997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.400488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1080/2163] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:34.401419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.401776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.409387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.411137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:34.441306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] MFA Rules not processed for user `0c3a70ba657a419990a55c448087859a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:34.446492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:34.492133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71324b3-616f-4c72-acaa-995a1d63d4f6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.555276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1084/2164] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:34.572760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de66c7-7faa-400a-8984-f2abf0ddffde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.572760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de66c7-7faa-400a-8984-f2abf0ddffde service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.572760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de66c7-7faa-400a-8984-f2abf0ddffde service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.572760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de66c7-7faa-400a-8984-f2abf0ddffde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.600635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.603223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:34.613784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:34.624702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab74032-214a-4811-9710-69f0f02fa752 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.630954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.631928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.632360 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.632869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.645552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.647018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.653415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de66c7-7faa-400a-8984-f2abf0ddffde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.656811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de66c7-7faa-400a-8984-f2abf0ddffde service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.661670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1085/2165] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:34.668615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1081/2166] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:34.677908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.759595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586722f4-ca7a-4b55-90f0-4c630e291817 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:34.765415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:34.766057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.766339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.766613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] PATH_INFO: `/v3/users/0c3a70ba657a419990a55c448087859a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.768038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c3a70ba657a419990a55c448087859a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:34.770081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:34.821859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:34.825031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c3a70ba657a419990a55c448087859a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:34.841866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c3a70ba657a419990a55c448087859a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:34.850065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c3a70ba657a419990a55c448087859a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:34.850760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c3a70ba657a419990a55c448087859a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:34.851495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c3a70ba657a419990a55c448087859a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:34.867881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1086/2167] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:34 2022] DELETE /identity/v3/users/0c3a70ba657a419990a55c448087859a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:34.872566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:34.872978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:34.873261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:34.895078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:34.896349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:34.977448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1082/2168] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:34.990089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:34.991567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:35.087481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471126b-9b25-48e6-8a0b-a21b96648ba8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:35.092293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea3395-906d-49dd-be37-d6b297b8c0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:35.093168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea3395-906d-49dd-be37-d6b297b8c0ec service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.093432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea3395-906d-49dd-be37-d6b297b8c0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.093715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea3395-906d-49dd-be37-d6b297b8c0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.106277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea3395-906d-49dd-be37-d6b297b8c0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:35.107903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea3395-906d-49dd-be37-d6b297b8c0ec service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:35.119067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1087/2169] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:35.450226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:35.456512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:35.471180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334ff5-c5dc-4e33-ae2e-7793d20b6dcd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:35.477618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:35.478483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.478770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.479050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] PATH_INFO: `/v3/projects/2706e2f2b8cf490f8cc2142ff5f59293` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.481123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2706e2f2b8cf490f8cc2142ff5f59293)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:35.483510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:35.560699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2706e2f2b8cf490f8cc2142ff5f59293', 'request_id': 'req-a052fcba-9bfb-42e7-b76b-4373c0b055fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:35.572848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2706e2f2b8cf490f8cc2142ff5f59293', 'request_id': 'req-a052fcba-9bfb-42e7-b76b-4373c0b055fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:35.581025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2706e2f2b8cf490f8cc2142ff5f59293', 'request_id': 'req-a052fcba-9bfb-42e7-b76b-4373c0b055fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:35.606761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] The token cache is being invalidate because project 2706e2f2b8cf490f8cc2142ff5f59293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:35.607670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:35.608444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a052fcba-9bfb-42e7-b76b-4373c0b055fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:35.621469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1083/2170] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:35 2022] DELETE /identity/v3/projects/2706e2f2b8cf490f8cc2142ff5f59293 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:35.628337 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.629370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.629983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.657146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:35.658377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:35.723923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1088/2171] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:35.732903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.733192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.733486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.753509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:35.754741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:35.764714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1084/2172] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:35.769180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:35.782693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a2d0-081b-4da7-82f5-2d2a42696b19 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:35.787904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:35.788456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.788740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.789018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.789735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:35.790042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:35.791629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:35.804454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1089/2173] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:17:35.808966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:35.818382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443238b-2df6-4aea-b3d0-cc0e8b479866 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:35.823329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:35.823939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.824245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.824550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.825181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:35.827895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:35.862800 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1085/2174] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:35 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:35.863666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:35.873865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381bb28-1677-42e2-bafb-b5092b03e4c9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:35.924766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:35.934894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:35.935509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.935787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.936058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.936730 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:35.938705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:35.960750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1090/2175] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:35 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:35.964967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:35.976278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930342ff-aafe-4792-b04b-df83e3be351e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:35.981223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:35.982006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:35.982205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:35.982481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:35.983166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:35.984802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:35.995253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1086/2176] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:35.999983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:36.010854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab4505-59c9-42d2-97c7-5899205cf64a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:36.016712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0b5c5-ddc9-42ca-a693-e5463502980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:36.017575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0b5c5-ddc9-42ca-a693-e5463502980b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:36.017855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0b5c5-ddc9-42ca-a693-e5463502980b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:36.018126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0b5c5-ddc9-42ca-a693-e5463502980b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:36.018770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0b5c5-ddc9-42ca-a693-e5463502980b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:36.020646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0b5c5-ddc9-42ca-a693-e5463502980b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:36.040050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1091/2177] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:36.042917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:36.053320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e31fb-3a79-405c-8baf-4a01b7e8d642 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:36.058363 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:36.059031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:36.059309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:36.059586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] PATH_INFO: `/v3/projects/6f327b5f45b1418fabd87a2ec619a7ce/users/cd929192bdc94af9b70e30344e02b488/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:36.072503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:36.079131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f327b5f45b1418fabd87a2ec619a7ce, user_id=cd929192bdc94af9b70e30344e02b488, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:36.082195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:36.108794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1087/2178] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:36 2022] PUT /identity/v3/projects/6f327b5f45b1418fabd87a2ec619a7ce/users/cd929192bdc94af9b70e30344e02b488/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:36.110370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:36.110840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:36.110932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:36.140929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] MFA Rules not processed for user `cd929192bdc94af9b70e30344e02b488`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:36.142422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:36.210316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1092/2179] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:17:36.222492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:36.224280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:36.294849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344140d-0e59-4868-a3ee-95ff3a2bb03a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:36.298449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeee5ac-e75b-438b-8bcb-529deb47badb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:36.299084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee5ac-e75b-438b-8bcb-529deb47badb service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:36.299342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee5ac-e75b-438b-8bcb-529deb47badb service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:36.299611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee5ac-e75b-438b-8bcb-529deb47badb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:36.307735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeee5ac-e75b-438b-8bcb-529deb47badb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:36.309453 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeee5ac-e75b-438b-8bcb-529deb47badb service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:36.318867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1088/2180] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:39.176000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:39.178627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:39.304032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c9638-22a8-4987-a35d-a6c35433b92c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:39.311107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:39.311913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:39.312387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:39.313093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] PATH_INFO: `/v3/users/00d2979da0f84f359fee4207c7a0b694` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:39.314623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d2979da0f84f359fee4207c7a0b694)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:39.316079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:39.393134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d2979da0f84f359fee4207c7a0b694'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:39.411111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:39.414348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d2979da0f84f359fee4207c7a0b694'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:39.431131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d2979da0f84f359fee4207c7a0b694'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:39.434919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d2979da0f84f359fee4207c7a0b694'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:39.435963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d2979da0f84f359fee4207c7a0b694'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:39.453646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1093/2181] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:39 2022] DELETE /identity/v3/users/00d2979da0f84f359fee4207c7a0b694 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:39.458216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:39.458785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:39.458785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:39.489695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:39.491217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:39.583771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1089/2182] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:39.594055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:39.595715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:39.667304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09dbb-9adc-43f5-84f7-8ca7675a9697 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:39.672157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:39.672799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:39.673074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:39.673342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:39.683570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:39.685843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:39.696274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1094/2183] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:40.050802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:40.054486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.084329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de57f-c71a-4d7e-aebc-4a182545f289 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:40.097753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:40.098570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.098944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.099537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] PATH_INFO: `/v3/projects/be2ee6fbdcd147e8baecb701df0849d4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.100997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2ee6fbdcd147e8baecb701df0849d4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:40.103794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:40.209991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2ee6fbdcd147e8baecb701df0849d4', 'request_id': 'req-4fb54594-3f39-4489-aa4b-70cda0891b0c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.224950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2ee6fbdcd147e8baecb701df0849d4', 'request_id': 'req-4fb54594-3f39-4489-aa4b-70cda0891b0c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.227861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:40.232783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.234209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2ee6fbdcd147e8baecb701df0849d4', 'request_id': 'req-4fb54594-3f39-4489-aa4b-70cda0891b0c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.248997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] The token cache is being invalidate because project be2ee6fbdcd147e8baecb701df0849d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:40.249302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.249705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba74d34-03f9-43d3-81e1-b8bc038373eb service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:40.250174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fb54594-3f39-4489-aa4b-70cda0891b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.252020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1090/2184] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:40 2022] DELETE /identity/v3/projects/be2ee6fbdcd147e8baecb701df0849d4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:40.264470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ca92-490e-4428-991e-a08313b5d4b6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.264470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ca92-490e-4428-991e-a08313b5d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.264470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ca92-490e-4428-991e-a08313b5d4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.297676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bb99ca92-490e-4428-991e-a08313b5d4b6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:40.298936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bb99ca92-490e-4428-991e-a08313b5d4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.335297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:40.375000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:40.376486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.376964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.385229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] PATH_INFO: `/v3/users/cd929192bdc94af9b70e30344e02b488` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.387254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd929192bdc94af9b70e30344e02b488)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:40.400389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:40.455877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1091/2185] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:40.457396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300df378-730b-438b-8902-a09d3b957f93 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.457711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300df378-730b-438b-8902-a09d3b957f93 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.458411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300df378-730b-438b-8902-a09d3b957f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.491845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-300df378-730b-438b-8902-a09d3b957f93 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:40.494657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-300df378-730b-438b-8902-a09d3b957f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.507952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1092/2186] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:40.512885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300df378-730b-438b-8902-a09d3b957f93 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:40.519080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd929192bdc94af9b70e30344e02b488'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.532842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300df378-730b-438b-8902-a09d3b957f93 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:40.542052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:40.542947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.543389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.543835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.545492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:40.546261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:40.547691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd929192bdc94af9b70e30344e02b488'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.548987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:40.563006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd929192bdc94af9b70e30344e02b488'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.564825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd929192bdc94af9b70e30344e02b488'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.566005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0b21af00-2413-423d-bfd5-c896eb138c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd929192bdc94af9b70e30344e02b488'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:40.577029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1093/2187] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:17:40.582520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:40.585388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:40.586477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.606081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1095/2188] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:40 2022] DELETE /identity/v3/users/cd929192bdc94af9b70e30344e02b488 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:40.607809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.608102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.608410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.624503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029083c-f5ab-4d1d-855e-718d9a5bbe1e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:40.656060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:40.660659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.759254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:40.760954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.760954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.760954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.761487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:40.763724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:40.803850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1096/2189] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:40.815829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:40.816717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1094/2190] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:40 2022] POST /identity/v3/projects => generated 375 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:40.818258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:40.820847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:40.920458 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9f4df-2574-483a-9c0a-9710b7297cce None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:40.933975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf085883-b08d-4639-8f16-39abaa7452e4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:40.943148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:40.944630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:40.945152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:40.945539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:40.946444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:40.949629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:40.984812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1095/2191] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:40 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:40.989798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.008606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2872d-359a-461d-8df3-e5d940304011 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.020981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.022035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.023711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.024240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.029392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.031578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.039722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:41.043423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.044300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.044585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.044858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.048427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1096/2192] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:41.052136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.058717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.061299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.066035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98c3c-34b9-4e49-84d9-96136acb2e59 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.073450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44484d38-5053-4124-8dac-136b75877f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.074228 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44484d38-5053-4124-8dac-136b75877f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.074512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44484d38-5053-4124-8dac-136b75877f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.074788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44484d38-5053-4124-8dac-136b75877f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.074914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1097/2193] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:41.075995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44484d38-5053-4124-8dac-136b75877f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.078938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44484d38-5053-4124-8dac-136b75877f72 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.093536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1097/2194] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:41.101328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.112542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceebe1-2663-4d10-8f5e-3f3df9d591cc service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.119298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.120123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.120439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.120716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] PATH_INFO: `/v3/projects/da3ae68b12514b26baadcb97459a9242/users/de4faf08b8514e7da29a1cf13b73e833/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.143514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3ae68b12514b26baadcb97459a9242, user_id=de4faf08b8514e7da29a1cf13b73e833, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.146576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.174091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1098/2195] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:41 2022] PUT /identity/v3/projects/da3ae68b12514b26baadcb97459a9242/users/de4faf08b8514e7da29a1cf13b73e833/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:41.186686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.186956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.187231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.242619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.245204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:41.259627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] MFA Rules not processed for user `de4faf08b8514e7da29a1cf13b73e833`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:41.261684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:41.299705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:41.340557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdb56d-1e8f-4ff7-ac7c-29618edb712d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.346866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.347614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.347963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.348332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] PATH_INFO: `/v3/users/faf58e78faae461fa0e7b75eb30c6c61` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.349632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf58e78faae461fa0e7b75eb30c6c61)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.351698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.367685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1098/2196] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:41.379821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.381963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:41.416071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf58e78faae461fa0e7b75eb30c6c61'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.443422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf58e78faae461fa0e7b75eb30c6c61'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.453013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf58e78faae461fa0e7b75eb30c6c61'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.453790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf58e78faae461fa0e7b75eb30c6c61'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.454341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf58e78faae461fa0e7b75eb30c6c61'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.464099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc133f-e18e-4b4f-8ad0-3e4bddd76a27 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.470422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.470936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.471215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.471485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.472135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1099/2197] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:41 2022] DELETE /identity/v3/users/faf58e78faae461fa0e7b75eb30c6c61 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:41.472736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.474152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:41.476692 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:41.492687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.494309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.507673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1099/2198] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:41.570819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874d8d-5773-4af9-a3fb-1e2c7c033d0c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.587263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.587263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.587263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.587263 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] PATH_INFO: `/v3/projects/6f327b5f45b1418fabd87a2ec619a7ce` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.587607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f327b5f45b1418fabd87a2ec619a7ce)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.587607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.651420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f327b5f45b1418fabd87a2ec619a7ce', 'request_id': 'req-b20b899a-ddb7-4e66-a743-68cf08130684'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.657526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:41.659667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f327b5f45b1418fabd87a2ec619a7ce', 'request_id': 'req-b20b899a-ddb7-4e66-a743-68cf08130684'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.673500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f327b5f45b1418fabd87a2ec619a7ce', 'request_id': 'req-b20b899a-ddb7-4e66-a743-68cf08130684'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.688732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] The token cache is being invalidate because project 6f327b5f45b1418fabd87a2ec619a7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:41.689031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.690044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b20b899a-ddb7-4e66-a743-68cf08130684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.692230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1100/2199] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:41 2022] DELETE /identity/v3/projects/6f327b5f45b1418fabd87a2ec619a7ce => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:41.729292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:41.732473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:41.775062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337188-2618-41bc-8f06-fa06a431f43d service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:41.882211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:41.882211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:41.882646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:41.882646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] PATH_INFO: `/v3/projects/b3cf862a38da43c081db5e171efe9459` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:41.884059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3cf862a38da43c081db5e171efe9459)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:41.886906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:41.925982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:41.957490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3cf862a38da43c081db5e171efe9459', 'request_id': 'req-2caa048f-0ec4-4b46-87ad-f16007769b3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.970677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3cf862a38da43c081db5e171efe9459', 'request_id': 'req-2caa048f-0ec4-4b46-87ad-f16007769b3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.978780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3cf862a38da43c081db5e171efe9459', 'request_id': 'req-2caa048f-0ec4-4b46-87ad-f16007769b3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.995809 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] The token cache is being invalidate because project b3cf862a38da43c081db5e171efe9459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:41.996025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:41.996759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2caa048f-0ec4-4b46-87ad-f16007769b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:42.014842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1100/2200] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:41 2022] DELETE /identity/v3/projects/b3cf862a38da43c081db5e171efe9459 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:42.021503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.021830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.022111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.073140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:42.077629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:42.181012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1101/2201] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:42.189481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.189771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.190048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.212134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:42.212134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:42.220240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1101/2202] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:42.224531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.235051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c164-2c85-4999-b4cc-cfc6ade382d3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.239727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.240380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.240661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.240934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.241593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:42.241896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.243526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.260180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1102/2203] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:42.266528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.277024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808ef27-b332-4b00-83da-3898aac6841e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.281819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.282422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.282695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.282961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.283588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.285497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.331514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1102/2204] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:42 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:42.335828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.346603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a48e1-1c8d-48c7-a76e-01441e859d79 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.408545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.409154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.409435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.409714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.410351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.412346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.436532 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1103/2205] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:42 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:42.440721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.457842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5b803-4bc9-4e9c-ac48-f956cb275d1f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.463587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.464414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.464982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.465090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.465947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.468720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.480019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1103/2206] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:42.485072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.495086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c73d5-f944-405c-8718-190987bd9fcd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.500916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0eb62-56fe-4bf7-abf4-09789720e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.501520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0eb62-56fe-4bf7-abf4-09789720e20a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.501796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0eb62-56fe-4bf7-abf4-09789720e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.502060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0eb62-56fe-4bf7-abf4-09789720e20a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.502691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0eb62-56fe-4bf7-abf4-09789720e20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.504332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0eb62-56fe-4bf7-abf4-09789720e20a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.516048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1104/2207] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:42.520509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.531238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48377acf-b916-43df-a883-bbba24c657e4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.535934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.536618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.537046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.537160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] PATH_INFO: `/v3/projects/867cf6bda53c41afa971ecd36c88e599/users/d0a1197f1c334123b07e7b66234e941b/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.553738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=867cf6bda53c41afa971ecd36c88e599, user_id=d0a1197f1c334123b07e7b66234e941b, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.556615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.572581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1104/2208] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:42 2022] PUT /identity/v3/projects/867cf6bda53c41afa971ecd36c88e599/users/d0a1197f1c334123b07e7b66234e941b/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:42.583059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.583339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.583611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.612359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] MFA Rules not processed for user `d0a1197f1c334123b07e7b66234e941b`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:42.613584 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:42.659363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:42.690039 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1105/2209] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:42.702203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:42.703888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:42.784294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4370-5b9e-4059-b5b4-cbfd758b0902 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:42.789775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6a34-e524-49fd-bdc2-d3ebfc962277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:42.790407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6a34-e524-49fd-bdc2-d3ebfc962277 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:42.790687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6a34-e524-49fd-bdc2-d3ebfc962277 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:42.790959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6a34-e524-49fd-bdc2-d3ebfc962277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:42.798912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6a34-e524-49fd-bdc2-d3ebfc962277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:42.800564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6a34-e524-49fd-bdc2-d3ebfc962277 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:42.812704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1105/2210] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:47.752594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:47.834307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75b087-66ab-409b-96f8-3ac2fef535ad None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:47.839121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:47.839763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:47.840043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:47.840351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] PATH_INFO: `/v3/users/d0a1197f1c334123b07e7b66234e941b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:47.841520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a1197f1c334123b07e7b66234e941b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:47.843464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:47.898087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a1197f1c334123b07e7b66234e941b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:47.914349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a1197f1c334123b07e7b66234e941b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:47.923718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a1197f1c334123b07e7b66234e941b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:47.924429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a1197f1c334123b07e7b66234e941b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:47.925190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a1197f1c334123b07e7b66234e941b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:47.940516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1106/2211] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:17:47 2022] DELETE /identity/v3/users/d0a1197f1c334123b07e7b66234e941b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:47.944816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:47.945560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:47.945560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:47.965316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:47.967075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:48.002612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:48.050832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1106/2212] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:48.061490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:48.062922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:48.135356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c4f57-8d87-4803-a3a3-5cbde6da9a14 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:48.139868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06dacc-309e-4759-9258-e0ab7885d7e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:48.140521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06dacc-309e-4759-9258-e0ab7885d7e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.140939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06dacc-309e-4759-9258-e0ab7885d7e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.141219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06dacc-309e-4759-9258-e0ab7885d7e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.153473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06dacc-309e-4759-9258-e0ab7885d7e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:48.154822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06dacc-309e-4759-9258-e0ab7885d7e0 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:48.165525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1107/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:17:48.471611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:48.473391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:48.491749 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9c952-e494-4b92-8f7b-0089605a7fbd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:48.496846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:48.497811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.498099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.498432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] PATH_INFO: `/v3/projects/867cf6bda53c41afa971ecd36c88e599` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.499790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867cf6bda53c41afa971ecd36c88e599)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:48.501992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:48.570011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867cf6bda53c41afa971ecd36c88e599', 'request_id': 'req-cb4e60c3-193b-4f65-8115-b60462f5ebe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:48.580376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867cf6bda53c41afa971ecd36c88e599', 'request_id': 'req-cb4e60c3-193b-4f65-8115-b60462f5ebe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:48.587486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867cf6bda53c41afa971ecd36c88e599', 'request_id': 'req-cb4e60c3-193b-4f65-8115-b60462f5ebe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:48.600824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] The token cache is being invalidate because project 867cf6bda53c41afa971ecd36c88e599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:17:48.601355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:48.602411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cb4e60c3-193b-4f65-8115-b60462f5ebe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:17:48.604358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1107/2214] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:17:48 2022] DELETE /identity/v3/projects/867cf6bda53c41afa971ecd36c88e599 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:48.612421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.613037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.613520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.627694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:48.647509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:48.649660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:48.718460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1108/2215] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:48.727223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.727694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.728155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.757478 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:48.758718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:48.767990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1108/2216] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:48.772352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:48.783399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57a97d-f82f-46e4-8569-14cb77e20cbf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:48.788088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:48.788749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.789029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.789297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.789951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:17:48.790251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:48.791874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:48.807939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1109/2217] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:17:48.814208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:48.824132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b932a1-c138-4afb-b4e0-57c69d6174ac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:48.829835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:48.830389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.830658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.830927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.831555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:48.833491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:48.866420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1109/2218] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:48 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:48.867614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:48.878747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1c4ac-effe-4029-afaf-efe028637ecf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:48.953276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:48.953804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:48.954083 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:48.954358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:48.954984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:48.956965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:48.983865 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1110/2219] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:48 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:17:48.985126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.007526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7f525-91a1-438c-a5bd-b985936e0dbd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.012464 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.013669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.013960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.014287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.015088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.016766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.028002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1110/2220] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:49.032652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.044843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2678-8ca0-4624-8abf-86529bd5b541 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.050082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960305b-80f9-42d1-9623-8bfdf5e172f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.050699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960305b-80f9-42d1-9623-8bfdf5e172f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.050979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960305b-80f9-42d1-9623-8bfdf5e172f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.051627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960305b-80f9-42d1-9623-8bfdf5e172f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.052132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960305b-80f9-42d1-9623-8bfdf5e172f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.053825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960305b-80f9-42d1-9623-8bfdf5e172f9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.068230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1111/2221] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:17:49.073750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.086839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f88a03-cebb-44a1-98bc-f24102421d45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.091983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.092789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.093072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.093346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] PATH_INFO: `/v3/projects/0a9b25154c7441338d75880469342ead/users/28b6cf0771c249c097e03a859d9fde32/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.121736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9b25154c7441338d75880469342ead, user_id=28b6cf0771c249c097e03a859d9fde32, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.125374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.152227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1111/2222] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:49 2022] PUT /identity/v3/projects/0a9b25154c7441338d75880469342ead/users/28b6cf0771c249c097e03a859d9fde32/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:49.171021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.171399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.171767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.214555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] MFA Rules not processed for user `28b6cf0771c249c097e03a859d9fde32`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:49.216056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:49.327919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:49.347430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1112/2223] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:49.350742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.470843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:49.491260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b2915-8c46-4a20-ab10-a7ddb3a36974 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.501787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.502513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.502866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.503218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.503907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.506461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.544122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1112/2224] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:17:49 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:49.545893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.567443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb2d0-2542-4eae-a283-b5d5abeace89 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.640820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.641477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.641754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.642027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.642917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.645058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.675549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1113/2225] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:17:49 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:17:49.676776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.688285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795149be-d27a-4675-80df-e59b41a9e004 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.694707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.694707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.694707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.694707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.695157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.696824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.708950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1113/2226] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:49.713397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.725324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3059a0-8b35-49bc-9823-90d47100c695 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.729949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.731031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.731303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.732329 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] PATH_INFO: `/v3/projects/b2afd4ecb9a94275b75293ffffa0a84e/users/63eec719c4594ce9a8b2a82ae8c762e8/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.753580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2afd4ecb9a94275b75293ffffa0a84e, user_id=63eec719c4594ce9a8b2a82ae8c762e8, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.756454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.772851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1114/2227] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:49 2022] PUT /identity/v3/projects/b2afd4ecb9a94275b75293ffffa0a84e/users/63eec719c4594ce9a8b2a82ae8c762e8/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:17:49.777175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.788959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12654b-44f5-42b0-a65e-4de64000d66d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.859129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad3bb5-d9d9-44df-9e97-467cf41ec4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.859856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3bb5-d9d9-44df-9e97-467cf41ec4ca admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.860134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3bb5-d9d9-44df-9e97-467cf41ec4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.860451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3bb5-d9d9-44df-9e97-467cf41ec4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.861101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad3bb5-d9d9-44df-9e97-467cf41ec4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.862796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad3bb5-d9d9-44df-9e97-467cf41ec4ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.877215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1114/2228] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:17:49.878370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:49.889705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e6cf-41d8-4abc-a064-a1c655a3c97c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:49.895307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:49.895993 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.896336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.896607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] PATH_INFO: `/v3/projects/b2afd4ecb9a94275b75293ffffa0a84e/users/63eec719c4594ce9a8b2a82ae8c762e8/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.898547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2afd4ecb9a94275b75293ffffa0a84e, user_id=63eec719c4594ce9a8b2a82ae8c762e8, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:49.901255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:49.915193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1115/2229] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:17:49 2022] PUT /identity/v3/projects/b2afd4ecb9a94275b75293ffffa0a84e/users/63eec719c4594ce9a8b2a82ae8c762e8/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:17:49.924038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:49.924458 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:49.924740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:49.960489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] MFA Rules not processed for user `63eec719c4594ce9a8b2a82ae8c762e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:17:49.960489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:50.046701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1115/2230] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:17:50.058241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:50.060594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:17:50.128691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87aa2-4f2c-426e-bff9-7fd4f91acf0f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:50.132509 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:50.133125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:50.133404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:50.133676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:50.145047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:50.146501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:50.157625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1116/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:17:50.243459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:17:50.256823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6ca7-4c4d-4a3d-ab6a-75a5c90ecdb7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:17:50.260570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb32f06-d99a-469a-a310-15130bca3754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:17:50.261171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb32f06-d99a-469a-a310-15130bca3754 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:17:50.261443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb32f06-d99a-469a-a310-15130bca3754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:17:50.261867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb32f06-d99a-469a-a310-15130bca3754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:17:50.338181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:17:50.354791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb32f06-d99a-469a-a310-15130bca3754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:17:50.356719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb32f06-d99a-469a-a310-15130bca3754 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:17:50.368605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1116/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:17:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:18:00.152631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:00.363414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:09.006600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:09.008060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:09.093237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8531d-b94c-4738-bd01-8b3b02bc1b21 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:09.098103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:09.098894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:09.099115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:09.099391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] PATH_INFO: `/v3/users/de4faf08b8514e7da29a1cf13b73e833` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:09.100594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4faf08b8514e7da29a1cf13b73e833)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:09.102563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:09.164003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4faf08b8514e7da29a1cf13b73e833'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:09.177732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4faf08b8514e7da29a1cf13b73e833'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:09.189248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4faf08b8514e7da29a1cf13b73e833'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:09.189248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4faf08b8514e7da29a1cf13b73e833'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:09.191520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4faf08b8514e7da29a1cf13b73e833'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:09.207214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1117/2233] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:18:09 2022] DELETE /identity/v3/users/de4faf08b8514e7da29a1cf13b73e833 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:18:09.211804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75eb1-efdf-4b43-a636-227ef824b415 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:09.212350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75eb1-efdf-4b43-a636-227ef824b415 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:09.212439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75eb1-efdf-4b43-a636-227ef824b415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:09.236748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0dc75eb1-efdf-4b43-a636-227ef824b415 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:09.238009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0dc75eb1-efdf-4b43-a636-227ef824b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:09.305390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1117/2234] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:18:09.325172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:09.326894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:09.391779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898f047-a2a3-479c-879f-559fdf606c6e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:09.395784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:09.396448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:09.396779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:09.397057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:09.407402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:09.408891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:09.419383 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1118/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:18:09.807664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:09.808322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:09.808804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:09.831398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] MFA Rules not processed for user `7ea15f54d31149ea948650cdd363d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:09.833050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:09.932399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1118/2236] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:09.933201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:09.934842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:09.949308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac02a26-af16-43ce-b91d-7ae03650858f service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:09.956007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:09.956707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:09.956995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:09.957367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] PATH_INFO: `/v3/projects/da3ae68b12514b26baadcb97459a9242` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:09.958782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3ae68b12514b26baadcb97459a9242)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:09.961008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:09.984535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:10.057980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3ae68b12514b26baadcb97459a9242', 'request_id': 'req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.067658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3ae68b12514b26baadcb97459a9242', 'request_id': 'req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.079557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3ae68b12514b26baadcb97459a9242', 'request_id': 'req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.102258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] The token cache is being invalidate because project da3ae68b12514b26baadcb97459a9242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:18:10.102561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.103392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655ad1a9-7fc5-495c-af1b-5ab8f2acae54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.109205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1119/2237] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:18:09 2022] DELETE /identity/v3/projects/da3ae68b12514b26baadcb97459a9242 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:18:10.819256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:10.820783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:10.835893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4de0d-565e-49ec-aa2d-88c69357d4d1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:10.910413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:10.911059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:10.911335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:10.911605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] PATH_INFO: `/v3/users/39b50bd816164229b8d4f9f416b40e40` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:10.912838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b50bd816164229b8d4f9f416b40e40)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:10.914817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:10.984092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b50bd816164229b8d4f9f416b40e40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.984827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b50bd816164229b8d4f9f416b40e40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.985444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b50bd816164229b8d4f9f416b40e40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:10.986812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:11.009644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b50bd816164229b8d4f9f416b40e40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.021750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b50bd816164229b8d4f9f416b40e40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.041808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1119/2238] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:18:10 2022] DELETE /identity/v3/users/39b50bd816164229b8d4f9f416b40e40 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:18:11.047449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.047738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.048010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.069536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:11.070752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:11.141013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1120/2239] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:11.150966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:11.152631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:11.228467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5952c1d-ff6e-42f9-8a2d-a898582e3eda admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:11.233084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ae026-9ec5-46a0-9101-f6a94db09223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:11.233857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae026-9ec5-46a0-9101-f6a94db09223 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.234140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae026-9ec5-46a0-9101-f6a94db09223 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.234415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae026-9ec5-46a0-9101-f6a94db09223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.248926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae026-9ec5-46a0-9101-f6a94db09223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:11.250369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae026-9ec5-46a0-9101-f6a94db09223 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:11.261863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1120/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:18:11.618104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:11.620067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:11.634557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa98b9f-cc51-407e-876e-c8402d927c5e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:11.641518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:11.642180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.642457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.642736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] PATH_INFO: `/v3/projects/224754f7d09c49198bb852f91f340e4a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.644129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224754f7d09c49198bb852f91f340e4a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:11.646358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:11.707545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224754f7d09c49198bb852f91f340e4a', 'request_id': 'req-4f34ce40-53e4-4ed1-8820-03df0f904d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.713904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224754f7d09c49198bb852f91f340e4a', 'request_id': 'req-4f34ce40-53e4-4ed1-8820-03df0f904d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.722762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224754f7d09c49198bb852f91f340e4a', 'request_id': 'req-4f34ce40-53e4-4ed1-8820-03df0f904d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.736928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] The token cache is being invalidate because project 224754f7d09c49198bb852f91f340e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:18:11.737223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.737895 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4f34ce40-53e4-4ed1-8820-03df0f904d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:11.739411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1121/2241] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:18:11 2022] DELETE /identity/v3/projects/224754f7d09c49198bb852f91f340e4a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:18:11.744368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.744721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.744955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.766246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:11.767496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:11.835887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1121/2242] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:18:11.845840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.846393 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.846930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.871547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:11.873453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:11.881743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1122/2243] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:11.886227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:11.896602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15120215-fd40-48b6-ab1e-4126dee3162f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:11.901281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:11.901884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.902161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.902432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.903084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:18:11.903393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:11.905162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:11.917711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1122/2244] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:18:11.922635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:11.932385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd08e7-c021-448c-aac3-ae68234dc118 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:11.937861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:11.938465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:11.938737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:11.939005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:11.939625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:11.941568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:11.964174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:11.972482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1123/2245] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:18:11 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:18:11.976790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:11.988411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d2c6-3a46-45c2-b37e-f80970cc48fe admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:12.065439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:12.074082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:12.076157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:12.076393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:12.076660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:12.077876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:12.080792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:12.105781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1123/2246] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:18:11 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:18:12.109871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:12.128409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd77c67-855e-4d05-8046-a3e11f627c7c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:12.138336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:12.139179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:12.139460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:12.139695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:12.140591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:12.142527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:12.157978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1124/2247] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:18:12.163033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:12.176393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add28c06-7ca5-4bea-b75d-9a6e83c27e57 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:12.182570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b503aa1-386b-47ad-98d0-f446d3cd4306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:12.183280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b503aa1-386b-47ad-98d0-f446d3cd4306 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:12.183629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b503aa1-386b-47ad-98d0-f446d3cd4306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:12.183967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b503aa1-386b-47ad-98d0-f446d3cd4306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:12.185334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b503aa1-386b-47ad-98d0-f446d3cd4306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:12.187191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b503aa1-386b-47ad-98d0-f446d3cd4306 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:12.199469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1124/2248] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:18:12.204677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:12.215905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababc3cc-effa-4c00-9261-a82cddb13b1e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:12.221884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:12.222737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:12.223090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:12.223464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] PATH_INFO: `/v3/projects/50a8fc499a8440a7947d5a4ae6d381f3/users/dd9ca98f61ec40bdbb89b5d4f793fac7/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:12.242002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a8fc499a8440a7947d5a4ae6d381f3, user_id=dd9ca98f61ec40bdbb89b5d4f793fac7, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:12.245874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:12.264032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1125/2249] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:18:12 2022] PUT /identity/v3/projects/50a8fc499a8440a7947d5a4ae6d381f3/users/dd9ca98f61ec40bdbb89b5d4f793fac7/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:18:12.275351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:12.275642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:12.275949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:12.311906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] MFA Rules not processed for user `dd9ca98f61ec40bdbb89b5d4f793fac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:12.313200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:12.394883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1125/2250] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:12.406406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:12.408168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:12.487000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ee231-d7a4-403f-afc7-cd2a90ee0759 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:12.492199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6cfd3-fa96-4069-8601-8cbb426c2c60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:12.492997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6cfd3-fa96-4069-8601-8cbb426c2c60 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:12.493280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6cfd3-fa96-4069-8601-8cbb426c2c60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:12.493554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6cfd3-fa96-4069-8601-8cbb426c2c60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:12.507248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6cfd3-fa96-4069-8601-8cbb426c2c60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:12.508772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6cfd3-fa96-4069-8601-8cbb426c2c60 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:12.520377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1126/2251] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:18:22.391559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:22.515154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:45.768120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:45.842811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99caaec9-deef-4a38-b86b-3725d6d8a56b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:45.847607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:45.848284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:45.848585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:45.848860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] PATH_INFO: `/v3/users/dd9ca98f61ec40bdbb89b5d4f793fac7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:45.850028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9ca98f61ec40bdbb89b5d4f793fac7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:45.851977 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:45.913066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9ca98f61ec40bdbb89b5d4f793fac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:45.913864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9ca98f61ec40bdbb89b5d4f793fac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:45.914512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9ca98f61ec40bdbb89b5d4f793fac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:45.930401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9ca98f61ec40bdbb89b5d4f793fac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:45.941027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9ca98f61ec40bdbb89b5d4f793fac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:45.962275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1126/2252] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:18:45 2022] DELETE /identity/v3/users/dd9ca98f61ec40bdbb89b5d4f793fac7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:18:45.967793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:45.968818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:45.968818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:45.994033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:45.995346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:46.079942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1127/2253] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:46.093208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:46.094709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:46.172795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a291-f394-44d8-a6f3-cf7028be648a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:46.177284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828674-6a30-4ba8-9862-97eee5c6a3f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:46.177914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828674-6a30-4ba8-9862-97eee5c6a3f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:46.178482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828674-6a30-4ba8-9862-97eee5c6a3f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:46.178761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828674-6a30-4ba8-9862-97eee5c6a3f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:46.190623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9828674-6a30-4ba8-9862-97eee5c6a3f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:46.192497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9828674-6a30-4ba8-9862-97eee5c6a3f6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:46.206427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1127/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:18:46.552218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:46.553733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:46.566998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371768f6-2a12-487b-9ffd-0c74cd256c49 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:46.572315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:46.572997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:46.573641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:46.573641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] PATH_INFO: `/v3/projects/50a8fc499a8440a7947d5a4ae6d381f3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:46.574891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a8fc499a8440a7947d5a4ae6d381f3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:46.577628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:46.645937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a8fc499a8440a7947d5a4ae6d381f3', 'request_id': 'req-f4d63ae0-3a2c-43e1-847d-255f169eb90d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:46.653008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a8fc499a8440a7947d5a4ae6d381f3', 'request_id': 'req-f4d63ae0-3a2c-43e1-847d-255f169eb90d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:46.662867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a8fc499a8440a7947d5a4ae6d381f3', 'request_id': 'req-f4d63ae0-3a2c-43e1-847d-255f169eb90d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:46.678445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] The token cache is being invalidate because project 50a8fc499a8440a7947d5a4ae6d381f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:18:46.678736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:46.679420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4d63ae0-3a2c-43e1-847d-255f169eb90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:46.681158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1128/2255] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:18:46 2022] DELETE /identity/v3/projects/50a8fc499a8440a7947d5a4ae6d381f3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:18:46.686661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:46.687079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:46.687427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:46.710879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:46.714155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:46.733776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:46.785059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1128/2256] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:46.794615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:46.794947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:46.795135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:46.821166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:46.822603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:46.839464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1129/2257] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:46.843293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:46.854735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589e497-3b13-4c5d-b6bb-f75f10cd71ad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:46.861156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:46.861804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:46.862079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:46.862349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:46.863003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:18:46.863303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:46.864963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:46.893744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1129/2258] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:18:46.898303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:46.909610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf269461-617d-4965-a51c-6ce20cfda461 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:46.918622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:46.920828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:46.921345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:46.921828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:46.922711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:46.925130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:46.975849 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1130/2259] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:18:46 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:18:46.978231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:46.990143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ada6-a695-4f99-931f-b1d60ad1910b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:47.101773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:47.102738 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:47.103025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:47.103316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:47.107745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:47.107745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:47.143052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1130/2260] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:18:46 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:18:47.147169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:47.160846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a71f06-9c27-4b0f-bb71-55d5793b3407 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:47.165721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:47.166343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:47.166622 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:47.166890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:47.167544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:47.169817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:47.181940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1131/2261] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:18:47.186108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:47.196777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05eba-b81f-4224-9651-83aa8af30be7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:47.203003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eedc7-b4f5-4833-a76d-e5436eb2ddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:47.203642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eedc7-b4f5-4833-a76d-e5436eb2ddd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:47.203924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eedc7-b4f5-4833-a76d-e5436eb2ddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:47.204220 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eedc7-b4f5-4833-a76d-e5436eb2ddd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:47.204881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eedc7-b4f5-4833-a76d-e5436eb2ddd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:47.206528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eedc7-b4f5-4833-a76d-e5436eb2ddd6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:47.219278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1131/2262] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:18:47.223369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:47.234072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22f955-4927-44a6-af03-1c3710875255 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:47.239114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:47.239789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:47.240066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:47.240370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] PATH_INFO: `/v3/projects/c14bc6c22cf74ac7b59d0c55e82c0829/users/ebfd685af8cd428bbed9a35ef0325f8b/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:47.259139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14bc6c22cf74ac7b59d0c55e82c0829, user_id=ebfd685af8cd428bbed9a35ef0325f8b, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:47.262218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:47.286586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1132/2263] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:18:47 2022] PUT /identity/v3/projects/c14bc6c22cf74ac7b59d0c55e82c0829/users/ebfd685af8cd428bbed9a35ef0325f8b/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:18:47.291572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607adf-9986-44bf-a61b-b50398b64461 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:47.291848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607adf-9986-44bf-a61b-b50398b64461 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:47.292123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607adf-9986-44bf-a61b-b50398b64461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:47.314653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6b607adf-9986-44bf-a61b-b50398b64461 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:47.315933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6b607adf-9986-44bf-a61b-b50398b64461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:47.423523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1132/2264] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:47.433640 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:47.436942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:47.469572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:47.533936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57d280-4c4d-49ba-92dd-c095e2d8e450 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:47.540343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:47.540972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:47.541251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:47.541523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:47.552426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:47.553875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:47.564672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1133/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:18:52.131336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.131908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.131908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.168217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] MFA Rules not processed for user `ebfd685af8cd428bbed9a35ef0325f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:52.169788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:52.197088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:52.274315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1133/2266] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:18:52.279471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:52.289715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa73f5-23b7-45bb-9e05-ef7899a78b54 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:52.297417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:52.298043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.298327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.298595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.299243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:52.301218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:52.346211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1134/2267] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:18:52 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:18:52.350263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:52.367925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f370b-724a-451a-96a4-5c5609e93326 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:52.443473 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:52.444063 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.444386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.444659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.445292 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:52.447428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:52.486047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1134/2268] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:18:52 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:18:52.491562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:52.505310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84525699-0e7b-4e80-8b4c-e42ab8bf2796 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:52.510891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:52.511664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.511769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.512041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.512913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:52.514573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:52.535389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1135/2269] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:18:52.536853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:52.553908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d96c77-1843-4423-93d1-2f2f22746663 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:52.559695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:52.560537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.560817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.561089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] PATH_INFO: `/v3/projects/26b16514d9854dfeba0ba85334246d05/users/6e8eb83d69f64a3cb6cf72c253fd97a2/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.579177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b16514d9854dfeba0ba85334246d05, user_id=6e8eb83d69f64a3cb6cf72c253fd97a2, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:52.581965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:52.599408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1135/2270] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:18:52 2022] PUT /identity/v3/projects/26b16514d9854dfeba0ba85334246d05/users/6e8eb83d69f64a3cb6cf72c253fd97a2/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:18:52.603396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:52.628014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a3b1-5147-47a0-98e0-5b1831cf23b1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:52.719014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517febd-97c0-4061-835e-431d8aa792aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:52.719699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517febd-97c0-4061-835e-431d8aa792aa admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.720053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517febd-97c0-4061-835e-431d8aa792aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.720425 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517febd-97c0-4061-835e-431d8aa792aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.721213 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517febd-97c0-4061-835e-431d8aa792aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:52.722977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517febd-97c0-4061-835e-431d8aa792aa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:52.739379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1136/2271] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:18:52.740624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:52.751599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491d708-403b-4022-9dc1-98e9fb19382f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:52.757093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:52.757874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:52.758165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:52.758435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] PATH_INFO: `/v3/projects/26b16514d9854dfeba0ba85334246d05/users/6e8eb83d69f64a3cb6cf72c253fd97a2/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:52.760500 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b16514d9854dfeba0ba85334246d05, user_id=6e8eb83d69f64a3cb6cf72c253fd97a2, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:52.763224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:52.775291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1136/2272] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:18:52 2022] PUT /identity/v3/projects/26b16514d9854dfeba0ba85334246d05/users/6e8eb83d69f64a3cb6cf72c253fd97a2/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:18:58.315841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fdee9-f56e-4594-a80e-5e8b8e9db27f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:58.316401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fdee9-f56e-4594-a80e-5e8b8e9db27f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:58.317592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fdee9-f56e-4594-a80e-5e8b8e9db27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:58.379889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dc7fdee9-f56e-4594-a80e-5e8b8e9db27f None None] MFA Rules not processed for user `6e8eb83d69f64a3cb6cf72c253fd97a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:58.381603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dc7fdee9-f56e-4594-a80e-5e8b8e9db27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:58.495931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1137/2273] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:58.503197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:58.508839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:58.596954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33105b-ce13-4c6d-a52d-02600dd3de31 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:58.602858 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:58.606839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:58.607477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:58.609467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:58.673488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:58.675110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaf63c-0eae-4be7-8fef-38781a835876 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:58.675524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaf63c-0eae-4be7-8fef-38781a835876 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:58.676142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaf63c-0eae-4be7-8fef-38781a835876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:58.720082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ebeaf63c-0eae-4be7-8fef-38781a835876 None None] MFA Rules not processed for user `976558170ae64a9cbbd6930e7c3b048a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:58.721565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ebeaf63c-0eae-4be7-8fef-38781a835876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:58.727692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:58.742237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:58.745736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:58.769981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1137/2274] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:18:58.843279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1138/2275] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:18:58.951844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:58.953343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.031211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:59.032537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:59.033044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:59.058845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:59.060366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.084428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272485-0cbe-43f9-a2ea-1da51024af0f service glance] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:59.091383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:59.092542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:59.093020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:59.093462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] PATH_INFO: `/v3/users/28b6cf0771c249c097e03a859d9fde32` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:59.095038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b6cf0771c249c097e03a859d9fde32)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:59.097825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:59.173017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1139/2276] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:18:59.207459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b6cf0771c249c097e03a859d9fde32'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:59.208217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b6cf0771c249c097e03a859d9fde32'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:59.208854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b6cf0771c249c097e03a859d9fde32'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:59.227330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b6cf0771c249c097e03a859d9fde32'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:59.240459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c5a440e-3421-4886-848f-f7202660ec8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b6cf0771c249c097e03a859d9fde32'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:18:59.249644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:59.253244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.269514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1138/2277] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:18:58 2022] DELETE /identity/v3/users/28b6cf0771c249c097e03a859d9fde32 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:18:59.272553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:59.272903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:59.273105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:59.303922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:18:59.305769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.384011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94d970-6a78-4ae7-a028-5eb8ddda9b6a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:59.392856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:59.432124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1139/2278] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:18:59.432124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:59.432429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.491740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:59.492540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:59.493072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:59.493229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:59.495302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.500970 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:18:59.507466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180d349-1f4b-4c76-aa91-aac6751164c6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:59.513568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:59.514649 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:59.515162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:59.515515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:59.520154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:18:59.604522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:59.605997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:59.606121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:59.608581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:59.624242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1140/2279] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:18:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 11:18:59.624886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:18:59.645099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1140/2280] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:18:59.646319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f3da0-53ac-4150-9e43-bbd2bdfdae6a service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:18:59.651183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:18:59.651875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:18:59.652501 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:18:59.652839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:18:59.685610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:18:59.687161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:18:59.717619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1141/2281] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:00.390223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:00.393748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:00.437217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9502cf-320a-4122-a994-7df641a14f7e service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:00.447516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:00.448223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:00.448490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:00.448770 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] PATH_INFO: `/v3/projects/0a9b25154c7441338d75880469342ead` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:00.453093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9b25154c7441338d75880469342ead)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:00.455079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:00.577803 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9b25154c7441338d75880469342ead', 'request_id': 'req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:00.589958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9b25154c7441338d75880469342ead', 'request_id': 'req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:00.602194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9b25154c7441338d75880469342ead', 'request_id': 'req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:00.619853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] The token cache is being invalidate because project 0a9b25154c7441338d75880469342ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:00.620569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:00.621607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:00.625370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1141/2282] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:00 2022] DELETE /identity/v3/projects/0a9b25154c7441338d75880469342ead => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:00.632452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:00.657666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5dac5-6e14-4743-a4c7-10308133a823 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:00.816779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:00.817644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:00.818023 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:00.818476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] PATH_INFO: `/v3/users/63eec719c4594ce9a8b2a82ae8c762e8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:00.829034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63eec719c4594ce9a8b2a82ae8c762e8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:00.831156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:00.964508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63eec719c4594ce9a8b2a82ae8c762e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.017920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63eec719c4594ce9a8b2a82ae8c762e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.031739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63eec719c4594ce9a8b2a82ae8c762e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.032656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63eec719c4594ce9a8b2a82ae8c762e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.033582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2bafe9ae-d028-4f07-9acc-00417e5ca934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63eec719c4594ce9a8b2a82ae8c762e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.071046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1142/2283] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:00 2022] DELETE /identity/v3/users/63eec719c4594ce9a8b2a82ae8c762e8 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:01.642684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:01.650579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:01.679123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34ee96-6d87-4bc6-9aa6-6b8ea8d5b6ac admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:01.707552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:01.783009 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:01.784218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:01.784756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:01.785331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] PATH_INFO: `/v3/projects/b2afd4ecb9a94275b75293ffffa0a84e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:01.787820 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2afd4ecb9a94275b75293ffffa0a84e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:01.790584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:01.860225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2afd4ecb9a94275b75293ffffa0a84e', 'request_id': 'req-7b34e85b-5943-4305-9b5d-708f35142c2f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.882941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2afd4ecb9a94275b75293ffffa0a84e', 'request_id': 'req-7b34e85b-5943-4305-9b5d-708f35142c2f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.892783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2afd4ecb9a94275b75293ffffa0a84e', 'request_id': 'req-7b34e85b-5943-4305-9b5d-708f35142c2f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.924768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] The token cache is being invalidate because project b2afd4ecb9a94275b75293ffffa0a84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:01.925351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.926561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7b34e85b-5943-4305-9b5d-708f35142c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:01.928828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1142/2284] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:01 2022] DELETE /identity/v3/projects/b2afd4ecb9a94275b75293ffffa0a84e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:01.936079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:01.936079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:01.936079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:02.027624 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:02.039773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:02.058539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:02.360436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1143/2285] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:02.360763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:02.361067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:02.361335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:02.404492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:02.405705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:02.425512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1143/2286] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:02.429583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:02.444542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c784ed-2ec7-456d-b719-a0a7423f38bd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:02.451178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:02.451850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:02.452121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:02.452433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:02.453092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:19:02.453444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:02.455079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:02.483691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1144/2287] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:19:02.484876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:02.510593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c21d-5f76-4fd5-967b-0ae6e9355ccb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:02.531103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:02.531729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:02.532004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:02.532316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:02.532948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:02.534865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:02.670306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1144/2288] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:02 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:02.673282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:02.705343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fde1f-b587-4123-b782-66c07f57ea2f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:02.869345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:02.869981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:02.870259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:02.870595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:02.871406 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:02.874350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:02.915883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1145/2289] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:02 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:02.917257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:02.933599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bddc1-99a5-4920-9db8-249d13fa0815 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:02.946074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:02.946941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:02.947296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:02.947646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:02.948843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:02.951518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:02.968535 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1145/2290] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:02.969805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:02.983335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908cc22-b880-40e2-9634-e9a7b702574a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:03.000860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9055e2-1ae8-4fd1-97a1-ae7b0fb9aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:03.002377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9055e2-1ae8-4fd1-97a1-ae7b0fb9aec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:03.003510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9055e2-1ae8-4fd1-97a1-ae7b0fb9aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:03.004018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9055e2-1ae8-4fd1-97a1-ae7b0fb9aec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:03.004935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9055e2-1ae8-4fd1-97a1-ae7b0fb9aec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:03.006820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9055e2-1ae8-4fd1-97a1-ae7b0fb9aec3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:03.022769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1146/2291] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:03.023750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:03.040157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3b970-26f2-4c2d-a1cd-78c39ff32669 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:03.055095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:03.056084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:03.056084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:03.056336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] PATH_INFO: `/v3/projects/afd7be166b08447b9a05e4cae0d14c25/users/a499895d0d6848e1b9e388ef1f494979/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:03.078965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd7be166b08447b9a05e4cae0d14c25, user_id=a499895d0d6848e1b9e388ef1f494979, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:03.082549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:03.138678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1146/2292] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:03 2022] PUT /identity/v3/projects/afd7be166b08447b9a05e4cae0d14c25/users/a499895d0d6848e1b9e388ef1f494979/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:03.149955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:03.150243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:03.150518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:03.203251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] MFA Rules not processed for user `a499895d0d6848e1b9e388ef1f494979`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:03.204809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:03.374143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1147/2293] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:03.379685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:03.591607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:03.643057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c54f1-5e4e-4020-9a0c-89fb0435b227 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:03.660385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:03.661703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:03.663527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:03.664467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:03.666148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:03.671680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:03.758395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1147/2294] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:03 2022] POST /identity/v3/projects => generated 379 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:03.759477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:03.778540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a27c43-f377-414f-9a04-6bc9fd8cf86b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:03.786670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:03.936972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:03.943162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:03.943490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:03.943712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:03.944557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:03.951003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:04.025785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1148/2295] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:03 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:04.028271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:04.062554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716c238-ee3c-4c7e-bf98-f87fb4556e5d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:04.075895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:04.076617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:04.076898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:04.077164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:04.077827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:04.079658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:04.114866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1148/2296] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:04.116297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:04.130497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc261412-1023-40e1-a10b-1fc8620b9f85 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:04.143274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:04.143966 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:04.144276 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:04.144808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] PATH_INFO: `/v3/projects/b1b1a039a7e5445996ff31a4d2c40c98/users/ccacaffcc1f94bb697af5d973dd71eaa/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:04.185204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b1a039a7e5445996ff31a4d2c40c98, user_id=ccacaffcc1f94bb697af5d973dd71eaa, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:04.195136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:04.227210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1149/2297] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:04 2022] PUT /identity/v3/projects/b1b1a039a7e5445996ff31a4d2c40c98/users/ccacaffcc1f94bb697af5d973dd71eaa/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:04.231714 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:04.246386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467331d-f542-41b7-9843-738435f5bda1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:04.422348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e036c-7ee7-4bdb-97a7-2418fd91dd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:04.423349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e036c-7ee7-4bdb-97a7-2418fd91dd7f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:04.423349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e036c-7ee7-4bdb-97a7-2418fd91dd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:04.423564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e036c-7ee7-4bdb-97a7-2418fd91dd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:04.428525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e036c-7ee7-4bdb-97a7-2418fd91dd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:04.430314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e036c-7ee7-4bdb-97a7-2418fd91dd7f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:04.452808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1149/2298] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:04.456654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:04.468204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617761de-154c-4f50-b93a-1e62eddfd214 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:04.483064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:04.483751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:04.484033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:04.484335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] PATH_INFO: `/v3/projects/b1b1a039a7e5445996ff31a4d2c40c98/users/ccacaffcc1f94bb697af5d973dd71eaa/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:04.491918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b1a039a7e5445996ff31a4d2c40c98, user_id=ccacaffcc1f94bb697af5d973dd71eaa, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:04.500261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:04.513137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1150/2299] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:04 2022] PUT /identity/v3/projects/b1b1a039a7e5445996ff31a4d2c40c98/users/ccacaffcc1f94bb697af5d973dd71eaa/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:04.524876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:04.525523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:04.526044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:04.634367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] MFA Rules not processed for user `ccacaffcc1f94bb697af5d973dd71eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:04.638721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:04.887127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1150/2300] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:04.888894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:04.890768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:05.117687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b294b52-f898-48ee-9239-759adb1c5634 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:05.131130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:05.131636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:05.132148 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:05.132565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:05.151377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:05.155616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:05.181140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1151/2301] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:05.415261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:05.416808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:05.500160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:05.582854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6895d-14e8-4609-96d8-5cac24ee0ee1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:05.590175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:05.597014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:05.597318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:05.599885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:05.603337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:05.630383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:05.632281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce45f97-8e05-4e2a-af20-339be3f62783 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:05.650451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1151/2302] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:05.965434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:06.070536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01743f-efdf-4f6d-a157-9489a88ebaf0 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:06.077245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:06.078292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:06.078752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:06.079183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] PATH_INFO: `/v3/users/a499895d0d6848e1b9e388ef1f494979` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:06.080726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a499895d0d6848e1b9e388ef1f494979)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:06.084091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:06.089615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:06.243226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a499895d0d6848e1b9e388ef1f494979'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:06.267408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a499895d0d6848e1b9e388ef1f494979'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:06.280766 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a499895d0d6848e1b9e388ef1f494979'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:06.281698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a499895d0d6848e1b9e388ef1f494979'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:06.282495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a499895d0d6848e1b9e388ef1f494979'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:06.297417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1152/2303] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:05 2022] DELETE /identity/v3/users/a499895d0d6848e1b9e388ef1f494979 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:06.302208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21467614-d3de-415d-902e-33942429ed34 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:06.302717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21467614-d3de-415d-902e-33942429ed34 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:06.302810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21467614-d3de-415d-902e-33942429ed34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:06.333152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-21467614-d3de-415d-902e-33942429ed34 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:06.334684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21467614-d3de-415d-902e-33942429ed34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:06.454485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1152/2304] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:06.455593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:06.457258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:06.568976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4183fa0-8139-4a3a-b0c6-ecbbc39b2523 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:06.578020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:06.582184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:06.582564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:06.582915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:06.599336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:06.601205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:06.650350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1153/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:19:07.088681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21467614-d3de-415d-902e-33942429ed34 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:07.091136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21467614-d3de-415d-902e-33942429ed34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:07.137105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21467614-d3de-415d-902e-33942429ed34 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:07.150844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:07.151792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:07.152085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:07.152384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] PATH_INFO: `/v3/projects/afd7be166b08447b9a05e4cae0d14c25` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:07.154479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd7be166b08447b9a05e4cae0d14c25)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:07.156709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:07.268516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd7be166b08447b9a05e4cae0d14c25', 'request_id': 'req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.284747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd7be166b08447b9a05e4cae0d14c25', 'request_id': 'req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.293149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd7be166b08447b9a05e4cae0d14c25', 'request_id': 'req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.306687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] The token cache is being invalidate because project afd7be166b08447b9a05e4cae0d14c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:07.307868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.308752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.312035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1153/2306] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:07 2022] DELETE /identity/v3/projects/afd7be166b08447b9a05e4cae0d14c25 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:07.317620 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:07.332913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138af9e-307a-46b5-9ecf-4afdc7df08b6 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:07.436871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:07.437923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:07.438338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:07.438896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] PATH_INFO: `/v3/users/ccacaffcc1f94bb697af5d973dd71eaa` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:07.440910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccacaffcc1f94bb697af5d973dd71eaa)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:07.445650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:07.511114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:07.551196 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccacaffcc1f94bb697af5d973dd71eaa'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.584524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccacaffcc1f94bb697af5d973dd71eaa'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.608988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccacaffcc1f94bb697af5d973dd71eaa'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.611658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccacaffcc1f94bb697af5d973dd71eaa'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.612391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d6c5fcf3-b151-4bd6-b3d0-6a071a714b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccacaffcc1f94bb697af5d973dd71eaa'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:07.634351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1154/2307] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:07 2022] DELETE /identity/v3/users/ccacaffcc1f94bb697af5d973dd71eaa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:08.009074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.011221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:08.043443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b36c9-8cd9-43a6-8094-ffe0f6b87170 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.104718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:08.119866 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.120611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.121090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.121543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] PATH_INFO: `/v3/projects/b1b1a039a7e5445996ff31a4d2c40c98` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.123732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b1a039a7e5445996ff31a4d2c40c98)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.125788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.190789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b1a039a7e5445996ff31a4d2c40c98', 'request_id': 'req-49bc666f-7f38-4527-8d7a-afd219fbcf03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:08.203493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b1a039a7e5445996ff31a4d2c40c98', 'request_id': 'req-49bc666f-7f38-4527-8d7a-afd219fbcf03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:08.211450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b1a039a7e5445996ff31a4d2c40c98', 'request_id': 'req-49bc666f-7f38-4527-8d7a-afd219fbcf03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:08.227578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] The token cache is being invalidate because project b1b1a039a7e5445996ff31a4d2c40c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:08.227578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:08.227980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-49bc666f-7f38-4527-8d7a-afd219fbcf03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:08.229674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1154/2308] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:08 2022] DELETE /identity/v3/projects/b1b1a039a7e5445996ff31a4d2c40c98 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:08.235181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.235494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.235775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.263477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:08.265045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:08.334533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1155/2309] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:19:08.342916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.343463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.343463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.370713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:08.372527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:08.381345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1155/2310] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:08.385829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.399158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa9021-24d5-42fb-9cd0-9bf28261a6fd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.404929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.405616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.406876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.407242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.408053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:19:08.411007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.412713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.428031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1156/2311] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:19:08.432037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.443472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad9135-9b18-4284-89fd-a0adba8ac3f5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.451074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.452263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.452600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.452882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.453533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.455494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.496410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1156/2312] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:08 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:08.500793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.516552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6be4-4564-496b-9eb5-32a260578767 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.602809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.603630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.604300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.605114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.606709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.610737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.666325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1157/2313] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:08 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:08.670712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.698066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2fa07-f350-42a0-8e95-28220c632547 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.707757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.708752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.709372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.709865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.710759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.712683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.731031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1157/2314] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:08.735782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.749897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1decfff-f3a2-4a4f-8bc4-cd42c199d936 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.755558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0102d-f89a-46a6-85f8-f31a03582f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.756412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0102d-f89a-46a6-85f8-f31a03582f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.757674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0102d-f89a-46a6-85f8-f31a03582f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.757674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0102d-f89a-46a6-85f8-f31a03582f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.757997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0102d-f89a-46a6-85f8-f31a03582f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.759727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0102d-f89a-46a6-85f8-f31a03582f99 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.774016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1158/2315] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:08.779910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:08.806850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723fc1-3c72-41d6-a08e-287a07bdf245 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:08.819436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:08.820178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.820651 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.820987 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] PATH_INFO: `/v3/projects/0aff12f39b9b4df59d52145f54f9d2cb/users/32b5206e68454a048d390c9e5f68a2ec/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.854489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aff12f39b9b4df59d52145f54f9d2cb, user_id=32b5206e68454a048d390c9e5f68a2ec, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:08.858593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:08.880284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1158/2316] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:08 2022] PUT /identity/v3/projects/0aff12f39b9b4df59d52145f54f9d2cb/users/32b5206e68454a048d390c9e5f68a2ec/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:08.894000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5581ef-a9fb-4645-8cf5-9c56baa98cdd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:08.894190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5581ef-a9fb-4645-8cf5-9c56baa98cdd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:08.894461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5581ef-a9fb-4645-8cf5-9c56baa98cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:08.919665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:08.938382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4a5581ef-a9fb-4645-8cf5-9c56baa98cdd None None] MFA Rules not processed for user `32b5206e68454a048d390c9e5f68a2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:08.939937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4a5581ef-a9fb-4645-8cf5-9c56baa98cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:09.051498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1159/2317] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:09.067581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:09.073078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:09.148731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0af46-196e-4c3a-82de-8ff72b9db3b3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:09.155492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890255af-e3da-4a32-820e-dfd07b0cd2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:09.156114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890255af-e3da-4a32-820e-dfd07b0cd2fc service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:09.156439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890255af-e3da-4a32-820e-dfd07b0cd2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:09.156715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890255af-e3da-4a32-820e-dfd07b0cd2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:09.167382 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890255af-e3da-4a32-820e-dfd07b0cd2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:09.169490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890255af-e3da-4a32-820e-dfd07b0cd2fc service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:09.180122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1159/2318] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:19:13.393101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:13.393682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:13.393682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:13.413919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] MFA Rules not processed for user `ebfd685af8cd428bbed9a35ef0325f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:13.415149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:13.479523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1160/2319] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:19.175399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:21.439897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:21.440544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:21.440544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:21.462732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:21.463960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:21.526608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1160/2320] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:19:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:21.539253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:21.541174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:21.623157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5129e7-485e-465d-8ff2-247ea71af035 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:21.628152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84072-660c-45d1-be37-3ddd3bc1bb85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:21.628792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84072-660c-45d1-be37-3ddd3bc1bb85 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:21.629073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84072-660c-45d1-be37-3ddd3bc1bb85 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:21.629341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84072-660c-45d1-be37-3ddd3bc1bb85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:21.638439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d84072-660c-45d1-be37-3ddd3bc1bb85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:21.640092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d84072-660c-45d1-be37-3ddd3bc1bb85 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:21.649124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1161/2321] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:19:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:19:31.517079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:31.645855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:41.993907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:42.080221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ffa28-b4cd-4757-a2fa-07b96109b380 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:42.084720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:42.085350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:42.085625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:42.085890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] PATH_INFO: `/v3/users/32b5206e68454a048d390c9e5f68a2ec` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:42.087002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b5206e68454a048d390c9e5f68a2ec)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:42.089119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:42.154230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b5206e68454a048d390c9e5f68a2ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.154931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b5206e68454a048d390c9e5f68a2ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.155537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b5206e68454a048d390c9e5f68a2ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.170351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b5206e68454a048d390c9e5f68a2ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.178093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b5206e68454a048d390c9e5f68a2ec'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.192816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1161/2322] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:41 2022] DELETE /identity/v3/users/32b5206e68454a048d390c9e5f68a2ec => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:42.197234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:42.197828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:42.197828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:42.218374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:42.219764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:42.294497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1162/2323] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:42.304572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:42.306570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:42.375446 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb3942-6f9c-4c92-970e-545ae3948265 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:42.379211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8da76-8a1a-4f80-859a-053f21e07ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:42.379823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8da76-8a1a-4f80-859a-053f21e07ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:42.380104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8da76-8a1a-4f80-859a-053f21e07ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:42.380411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8da76-8a1a-4f80-859a-053f21e07ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:42.390619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8da76-8a1a-4f80-859a-053f21e07ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:42.392110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8da76-8a1a-4f80-859a-053f21e07ab7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:42.403471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1162/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:19:42.723664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:42.723664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:42.738100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fbb5c-8797-4dcc-a300-6e17b5176245 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:42.745591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:42.746527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:42.747515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:42.747981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] PATH_INFO: `/v3/projects/0aff12f39b9b4df59d52145f54f9d2cb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:42.752675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aff12f39b9b4df59d52145f54f9d2cb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:42.758301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:42.829177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aff12f39b9b4df59d52145f54f9d2cb', 'request_id': 'req-0aeeae4e-a092-4b0b-8167-3d9899433f96'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.836531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aff12f39b9b4df59d52145f54f9d2cb', 'request_id': 'req-0aeeae4e-a092-4b0b-8167-3d9899433f96'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.847530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aff12f39b9b4df59d52145f54f9d2cb', 'request_id': 'req-0aeeae4e-a092-4b0b-8167-3d9899433f96'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.860267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] The token cache is being invalidate because project 0aff12f39b9b4df59d52145f54f9d2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:42.860582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.861536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aeeae4e-a092-4b0b-8167-3d9899433f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:42.863272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1163/2325] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:42 2022] DELETE /identity/v3/projects/0aff12f39b9b4df59d52145f54f9d2cb => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:42.869338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:42.870016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:42.870622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:42.905126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:42.907369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:42.940427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:43.001393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1163/2326] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:43.017124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.017124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.017124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.040256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:43.041555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:43.050461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1164/2327] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:43.054763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.065705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450c12-4702-4f33-af6b-b726d30a3405 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.070406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.071052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.071333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.071607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.072300 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:19:43.072612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.074235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.086856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1164/2328] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:19:43.091998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.102109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaf268-3aea-4531-a39e-8cec83b315cb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.108152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.108804 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.109122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.109361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.109991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.111910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.144635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1165/2329] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:43.151463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.164901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee48f8-3775-4477-b035-42c7648eec8e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.239167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.239783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.240064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.240366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.241002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.242985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.266285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1165/2330] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:43.270633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.281621 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931bac-59ca-4366-8a53-6bd6c81654bb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.286429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.287052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.287334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.287609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.288287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.289913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.301578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1166/2331] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:43.305493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.316991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44260af-75b7-4af4-be02-8427c0f1d53e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.321903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c6457-32bf-4b2b-b9da-14d81af1c5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.322543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c6457-32bf-4b2b-b9da-14d81af1c5af admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.322792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c6457-32bf-4b2b-b9da-14d81af1c5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.323064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c6457-32bf-4b2b-b9da-14d81af1c5af admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.323702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c6457-32bf-4b2b-b9da-14d81af1c5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.325648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c6457-32bf-4b2b-b9da-14d81af1c5af admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.336856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1166/2332] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:43.341453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.359165 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167616-6b7d-44f7-8590-e8868969cefc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.364317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.364992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.365270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.365543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] PATH_INFO: `/v3/projects/4a955dc18e484cd5b96623298a3b88d2/users/cf6ca2abd0c94a7a93a0c54faf168801/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.386099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a955dc18e484cd5b96623298a3b88d2, user_id=cf6ca2abd0c94a7a93a0c54faf168801, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.389529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.408563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1167/2333] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:43 2022] PUT /identity/v3/projects/4a955dc18e484cd5b96623298a3b88d2/users/cf6ca2abd0c94a7a93a0c54faf168801/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:43.420761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.421051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.421330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.454561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] MFA Rules not processed for user `cf6ca2abd0c94a7a93a0c54faf168801`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:43.455845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:43.529524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1167/2334] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:43.534012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.557571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:43.605140 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc85c51-0d66-4182-aad9-84c0fcdebabe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.611101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.611751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.612455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.612455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.613138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.615100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.651169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1168/2335] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:43.652450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.664495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cce4d-1c5c-4758-bbea-27aeb3cfe809 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.679454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:43.733674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.734296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.734576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.734846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.735490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.738887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.759940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1168/2336] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:43.764386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.775107 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5fd79-03c2-4826-8841-aee5e05df3ea admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.781024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.781645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.781922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.782185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.782839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.784464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.798233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1169/2337] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:19:43.802391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.812962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dae831-0175-4134-ab96-4d3b103de5f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.817759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56562a72-0ca1-4136-9a96-8399bd81d044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.818367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56562a72-0ca1-4136-9a96-8399bd81d044 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.818645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56562a72-0ca1-4136-9a96-8399bd81d044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.818918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56562a72-0ca1-4136-9a96-8399bd81d044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.819589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56562a72-0ca1-4136-9a96-8399bd81d044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.821234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56562a72-0ca1-4136-9a96-8399bd81d044 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.832257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1169/2338] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:43.836555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:43.847490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38833-b090-47ab-84b4-97c485bf764f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:43.852347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:43.853015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.853295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.853574 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] PATH_INFO: `/v3/projects/762f3939bfb94189b7c017f075672a97/users/ee363acbc01e4039a3cb058497442cb2/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.870554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762f3939bfb94189b7c017f075672a97, user_id=ee363acbc01e4039a3cb058497442cb2, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:43.874577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:43.891115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1170/2339] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:43 2022] PUT /identity/v3/projects/762f3939bfb94189b7c017f075672a97/users/ee363acbc01e4039a3cb058497442cb2/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:43.901884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:43.902163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:43.902438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:43.933435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] MFA Rules not processed for user `ee363acbc01e4039a3cb058497442cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:43.934994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:44.013117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1170/2340] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:44.017578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.095820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cd88f-c34d-4e58-8f94-729ebf128521 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.101280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.102026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.102372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.102705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.103537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.107580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.141015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1171/2341] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:44 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:19:44.143705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.155503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20112829-8e06-47e2-ac1d-4d5909ad66db None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.228054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.228917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.229293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.229637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.230334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.233052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.259006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1171/2342] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:44 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:44.263875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.275394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd42ef0-07bb-4b46-abc0-fef8fe8be581 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.283171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.283798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.284074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.284375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.285019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.286669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.304339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1172/2343] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:44.305760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.317558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e68dd4-a82a-4993-95b0-eabe854f5271 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.322242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.322906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.323181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.323449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] PATH_INFO: `/v3/projects/7abfb602c483408aad34c9420d04159f/users/a8a834e3e83941b1997f3fa1027ea554/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.338630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:44.345084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abfb602c483408aad34c9420d04159f, user_id=a8a834e3e83941b1997f3fa1027ea554, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.348593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.368258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1172/2344] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:44 2022] PUT /identity/v3/projects/7abfb602c483408aad34c9420d04159f/users/a8a834e3e83941b1997f3fa1027ea554/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:44.372834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.383499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522a45d-6656-4d33-b635-8d2f9057a217 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.431313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:44.448017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58180917-8506-4a48-8114-8727828fafe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.448739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180917-8506-4a48-8114-8727828fafe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.449014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180917-8506-4a48-8114-8727828fafe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.449286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180917-8506-4a48-8114-8727828fafe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.450694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58180917-8506-4a48-8114-8727828fafe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.452328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58180917-8506-4a48-8114-8727828fafe2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.462609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1173/2345] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:44.467263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.477863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8baae-39cc-41cf-9847-e284785d557a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.483087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.483754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.484023 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.484339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] PATH_INFO: `/v3/projects/7abfb602c483408aad34c9420d04159f/users/a8a834e3e83941b1997f3fa1027ea554/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.486254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abfb602c483408aad34c9420d04159f, user_id=a8a834e3e83941b1997f3fa1027ea554, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.488954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.500581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1173/2346] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:44 2022] PUT /identity/v3/projects/7abfb602c483408aad34c9420d04159f/users/a8a834e3e83941b1997f3fa1027ea554/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:44.510805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.511081 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.511396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.541226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] MFA Rules not processed for user `a8a834e3e83941b1997f3fa1027ea554`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:44.542473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:44.614680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1174/2347] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:44.625586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:44.627309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:44.689017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f325fb-e691-49c2-b62e-a71a0ec3abfa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:44.692883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:44.693525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:44.693844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:44.694114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:44.755121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:44.756688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:44.768049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1174/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:46.316613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:46.318038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:46.330637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0241a-355c-4624-99ba-3538ecf8ffa8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:46.336266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:46.336266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:46.336266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:46.336266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:46.400761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:46.403859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:46.413347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1175/2349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:47.018705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:47.028727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c55a-069c-42c0-a8d1-acd6c5ba02f2 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:47.032541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8b942-a4b2-467b-b4d7-2ccc383e328a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:47.033152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8b942-a4b2-467b-b4d7-2ccc383e328a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:47.033427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8b942-a4b2-467b-b4d7-2ccc383e328a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:47.034220 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8b942-a4b2-467b-b4d7-2ccc383e328a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:47.043419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8b942-a4b2-467b-b4d7-2ccc383e328a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:47.044965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8b942-a4b2-467b-b4d7-2ccc383e328a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:47.054791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1175/2350] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:52.042659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:52.107806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7aa2f-fff3-443c-a69e-1ea60b6af2d6 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:52.112848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:52.113487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:52.113792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:52.114060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] PATH_INFO: `/v3/users/cf6ca2abd0c94a7a93a0c54faf168801` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:52.115206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6ca2abd0c94a7a93a0c54faf168801)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:52.117229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:52.123915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:52.176899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6ca2abd0c94a7a93a0c54faf168801'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.191011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6ca2abd0c94a7a93a0c54faf168801'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.198996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6ca2abd0c94a7a93a0c54faf168801'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.199711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6ca2abd0c94a7a93a0c54faf168801'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.200638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6ca2abd0c94a7a93a0c54faf168801'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.215924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1176/2351] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:52 2022] DELETE /identity/v3/users/cf6ca2abd0c94a7a93a0c54faf168801 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:52.221250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:52.221759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:52.221856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:52.245237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:52.246686 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:52.320992 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1176/2352] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:52.332910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:52.334390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:52.408869 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321ccad-2ab3-4b3c-a9e2-7c24aaef0baf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:52.412592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:52.413211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:52.413492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:52.413762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:52.423751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:52.425347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:52.435530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1177/2353] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:19:52.722258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:52.723823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:52.738180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93189a8-3439-4d68-a38b-7117b5eee941 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:52.749913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:52.749913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:52.749913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:52.749913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] PATH_INFO: `/v3/projects/4a955dc18e484cd5b96623298a3b88d2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:52.750411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a955dc18e484cd5b96623298a3b88d2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:52.751724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:52.769650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:52.804255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a955dc18e484cd5b96623298a3b88d2', 'request_id': 'req-6eca3655-53ad-4f40-84c4-b1c071fb63e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.812011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a955dc18e484cd5b96623298a3b88d2', 'request_id': 'req-6eca3655-53ad-4f40-84c4-b1c071fb63e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.818655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a955dc18e484cd5b96623298a3b88d2', 'request_id': 'req-6eca3655-53ad-4f40-84c4-b1c071fb63e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.831923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] The token cache is being invalidate because project 4a955dc18e484cd5b96623298a3b88d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:52.832240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.832932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.834420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1177/2354] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:52 2022] DELETE /identity/v3/projects/4a955dc18e484cd5b96623298a3b88d2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:52.838542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:52.849120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc52a6-c93e-4a26-9ee3-115c5c77d05e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:52.907298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:52.907923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:52.908217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:52.908492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] PATH_INFO: `/v3/users/ee363acbc01e4039a3cb058497442cb2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:52.909644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee363acbc01e4039a3cb058497442cb2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:52.911549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:52.941673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:52.961601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee363acbc01e4039a3cb058497442cb2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.978522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee363acbc01e4039a3cb058497442cb2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.987296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee363acbc01e4039a3cb058497442cb2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.987928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee363acbc01e4039a3cb058497442cb2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:52.988580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee363acbc01e4039a3cb058497442cb2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.001507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1178/2355] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:52 2022] DELETE /identity/v3/users/ee363acbc01e4039a3cb058497442cb2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:53.289011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:53.290606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:53.308763 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3655-53ad-4f40-84c4-b1c071fb63e8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:53.381443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:53.382223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:53.382519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:53.382780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] PATH_INFO: `/v3/projects/762f3939bfb94189b7c017f075672a97` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:53.383930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762f3939bfb94189b7c017f075672a97)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:53.385897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:53.448696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762f3939bfb94189b7c017f075672a97', 'request_id': 'req-7312bad1-479e-4736-b4e9-2e8a78485fd3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.458103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762f3939bfb94189b7c017f075672a97', 'request_id': 'req-7312bad1-479e-4736-b4e9-2e8a78485fd3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.466256 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762f3939bfb94189b7c017f075672a97', 'request_id': 'req-7312bad1-479e-4736-b4e9-2e8a78485fd3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.478501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] The token cache is being invalidate because project 762f3939bfb94189b7c017f075672a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:53.478785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.479445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.483375 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1178/2356] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:53 2022] DELETE /identity/v3/projects/762f3939bfb94189b7c017f075672a97 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:53.486001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:53.493911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:53.503628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3a4e5-6c28-4c37-b2d4-9bc19851988b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:53.568544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:53.569190 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:53.569474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:53.569796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] PATH_INFO: `/v3/users/a8a834e3e83941b1997f3fa1027ea554` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:53.570941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a834e3e83941b1997f3fa1027ea554)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:53.572961 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:53.634891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a834e3e83941b1997f3fa1027ea554'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.649707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a834e3e83941b1997f3fa1027ea554'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.657892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a834e3e83941b1997f3fa1027ea554'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.658555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a834e3e83941b1997f3fa1027ea554'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.659171 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-797de6db-b43f-47fd-8d69-0e73d69c5d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a834e3e83941b1997f3fa1027ea554'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:53.677690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1179/2357] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:19:53 2022] DELETE /identity/v3/users/a8a834e3e83941b1997f3fa1027ea554 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:19:53.995524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:53.998058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:54.016293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312bad1-479e-4736-b4e9-2e8a78485fd3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.053786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:54.085113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.085741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.086006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.086298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] PATH_INFO: `/v3/projects/7abfb602c483408aad34c9420d04159f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.088385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7abfb602c483408aad34c9420d04159f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.090928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.142557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7abfb602c483408aad34c9420d04159f', 'request_id': 'req-b15cc48c-3ed4-404a-b581-469243aa3149'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:54.151237 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7abfb602c483408aad34c9420d04159f', 'request_id': 'req-b15cc48c-3ed4-404a-b581-469243aa3149'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:54.158025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7abfb602c483408aad34c9420d04159f', 'request_id': 'req-b15cc48c-3ed4-404a-b581-469243aa3149'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:54.170592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] The token cache is being invalidate because project 7abfb602c483408aad34c9420d04159f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:19:54.170892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:54.171562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b15cc48c-3ed4-404a-b581-469243aa3149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:19:54.173094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1179/2358] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:19:53 2022] DELETE /identity/v3/projects/7abfb602c483408aad34c9420d04159f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:54.178074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.178369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.178669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.203668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:54.204949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:54.225591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:19:54.270420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1180/2359] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:54.281136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.281595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.282025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.303310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:54.304521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:54.314144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1180/2360] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:54.318120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.330208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687653e-999d-418f-bc0c-64c1c6afecc7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.334903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.335506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.335779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.336046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.336715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:19:54.337011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.338596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.354824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1181/2361] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:19:54.356028 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.366480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab93c62-0af0-4619-87c4-abbf0c254599 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.372350 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.373200 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.373516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.373794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.375144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.377179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.416154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1181/2362] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:19:54 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:54.420721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.431028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e88e3-8d7a-4f76-a671-c8cdec730d0b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.495403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.496117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.496301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.497105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.497885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.499807 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.524087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1182/2363] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:19:54 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:19:54.525553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.537856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b936fe-19d2-49f4-b62b-4831a6b6fc0f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.543084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.544331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.544619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.544888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.545536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.547168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.557506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1182/2364] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:54.562447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.572550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ce7bd-0e19-40d8-bb9e-4365b2d488fe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.577441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afbcc2-cd0e-4a6e-bfe9-190a5aef9b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.578068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afbcc2-cd0e-4a6e-bfe9-190a5aef9b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.578349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afbcc2-cd0e-4a6e-bfe9-190a5aef9b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.578623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afbcc2-cd0e-4a6e-bfe9-190a5aef9b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.579262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afbcc2-cd0e-4a6e-bfe9-190a5aef9b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.580906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afbcc2-cd0e-4a6e-bfe9-190a5aef9b5f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.591598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1183/2365] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:19:54.595748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.605731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebaa21-ba02-42d4-903f-8190b7cb26ff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.611105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.611744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.612026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.612345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] PATH_INFO: `/v3/projects/59763857f4474e8481d116d7f2d7a118/users/8b11ffd584644b71afd2edaccae10c57/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.627630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59763857f4474e8481d116d7f2d7a118, user_id=8b11ffd584644b71afd2edaccae10c57, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.630334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.643789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1183/2366] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:19:54 2022] PUT /identity/v3/projects/59763857f4474e8481d116d7f2d7a118/users/8b11ffd584644b71afd2edaccae10c57/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:19:54.654266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.654551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.654823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.682340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] MFA Rules not processed for user `8b11ffd584644b71afd2edaccae10c57`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:19:54.683549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:54.751345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1184/2367] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:19:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:19:54.763051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:19:54.764874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:19:54.827415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34df0-d532-4bd3-bf18-856acdb2a3a8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:19:54.831304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58d0c1-ed54-470b-b874-266bbb4842a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:19:54.831909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58d0c1-ed54-470b-b874-266bbb4842a9 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:19:54.832210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58d0c1-ed54-470b-b874-266bbb4842a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:19:54.832483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58d0c1-ed54-470b-b874-266bbb4842a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:19:54.841618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58d0c1-ed54-470b-b874-266bbb4842a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:19:54.843058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58d0c1-ed54-470b-b874-266bbb4842a9 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:19:54.852438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1184/2368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:19:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:20:01.683541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:01.685812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:01.709153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:20:01.746246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66413dcb-19d4-45dd-93ac-0d5b7e84550f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:01.750689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:01.751302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:01.751582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:01.751861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:01.753613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:01.821710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:01.823145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:01.833356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1185/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:20:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:20:04.848268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:20:11.829042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=41 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:20:44.768878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea5e21-095f-42d9-9a07-295364fb4868 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:44.769417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea5e21-095f-42d9-9a07-295364fb4868 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:44.769417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea5e21-095f-42d9-9a07-295364fb4868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:44.788366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a8ea5e21-095f-42d9-9a07-295364fb4868 None None] MFA Rules not processed for user `d045f032dbfb450aa9c087ada1637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:20:44.789568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a8ea5e21-095f-42d9-9a07-295364fb4868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:44.843395 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1185/2370] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:20:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:20:47.361702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:47.463494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d93f-e7ca-444e-b292-73b918d32c1b service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:47.470079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:47.471207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:47.471652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:47.472229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] PATH_INFO: `/v3/users/8b11ffd584644b71afd2edaccae10c57` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:47.473894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b11ffd584644b71afd2edaccae10c57)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:47.476521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:47.539937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b11ffd584644b71afd2edaccae10c57'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:47.556414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b11ffd584644b71afd2edaccae10c57'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:47.567040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b11ffd584644b71afd2edaccae10c57'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:47.567860 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b11ffd584644b71afd2edaccae10c57'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:47.568592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b11ffd584644b71afd2edaccae10c57'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:47.582834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1186/2371] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:20:47 2022] DELETE /identity/v3/users/8b11ffd584644b71afd2edaccae10c57 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:20:47.587434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:47.587973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:47.588098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:47.610380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:20:47.611708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:47.692825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1186/2372] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:20:47.703809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:47.705316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:47.772947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4f35a-ad98-4f2e-902f-b49e55d97df3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:47.779654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8bd2e-4442-47d8-ba2f-aa20c5c9c474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:47.780328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bd2e-4442-47d8-ba2f-aa20c5c9c474 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:47.780711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bd2e-4442-47d8-ba2f-aa20c5c9c474 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:47.780990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bd2e-4442-47d8-ba2f-aa20c5c9c474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:47.792002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bd2e-4442-47d8-ba2f-aa20c5c9c474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:47.794342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bd2e-4442-47d8-ba2f-aa20c5c9c474 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:47.805866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1187/2373] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:20:48.124730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.126170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:48.138338 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59f3b5-43bd-402f-8f1e-4ada60ca3a56 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.154053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.154715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.154994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.155381 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] PATH_INFO: `/v3/projects/59763857f4474e8481d116d7f2d7a118` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.156468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59763857f4474e8481d116d7f2d7a118)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.158391 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.212337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59763857f4474e8481d116d7f2d7a118', 'request_id': 'req-35e3c916-c1aa-4674-8fe1-8255c9216c17'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:48.221327 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59763857f4474e8481d116d7f2d7a118', 'request_id': 'req-35e3c916-c1aa-4674-8fe1-8255c9216c17'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:48.228958 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59763857f4474e8481d116d7f2d7a118', 'request_id': 'req-35e3c916-c1aa-4674-8fe1-8255c9216c17'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:48.240759 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] The token cache is being invalidate because project 59763857f4474e8481d116d7f2d7a118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:20:48.241051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:48.241742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35e3c916-c1aa-4674-8fe1-8255c9216c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:20:48.243270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1187/2374] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:20:48 2022] DELETE /identity/v3/projects/59763857f4474e8481d116d7f2d7a118 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:20:48.248992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.249357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.249666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.270731 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:20:48.271979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:48.296499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:20:48.358399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1188/2375] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:20:48.367059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.367340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.367635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.392368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:20:48.393606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:48.407975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1188/2376] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:20:48.409932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.421529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369749e9-2afe-4546-a4ea-59220ab500a4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.426479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.427099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.427525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.427798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.428485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:20:48.428786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.430396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.444298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1189/2377] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:20:48.449856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.461520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e956c9-85c2-4800-9815-3df6aa73cb47 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.467757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.468503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.468855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.469196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.469893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.471897 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.494540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:20:48.502856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1189/2378] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:20:48 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:20:48.506570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.517463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718cb09-5582-4264-97ac-81f92cc65a84 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.594666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.595318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.595591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.595866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.596537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.598466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.626675 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.627585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1190/2379] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:20:48 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:20:48.641283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964e920-bdde-4899-830b-64a7498b18e9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.647569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.648371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.648744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.649114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.649993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.652050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.670133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1190/2380] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:20:48.671395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.681613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a98de-b64e-40ba-8618-10c59c5f6f65 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.687530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51a85-df7a-4c68-8a86-4a2c0add5af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.688141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51a85-df7a-4c68-8a86-4a2c0add5af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.688591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51a85-df7a-4c68-8a86-4a2c0add5af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.688874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51a85-df7a-4c68-8a86-4a2c0add5af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.689520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b51a85-df7a-4c68-8a86-4a2c0add5af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.691133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b51a85-df7a-4c68-8a86-4a2c0add5af6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.702462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1191/2381] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:20:48.706784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.717291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f6563-395f-471d-b1a3-cc3d79482800 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.722090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.722748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.723037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.723294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] PATH_INFO: `/v3/projects/62556a3d77ac46cd86af53cb22884a4c/users/09d3e38998394634a141e5d34bf21dda/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.739840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62556a3d77ac46cd86af53cb22884a4c, user_id=09d3e38998394634a141e5d34bf21dda, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.742617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.760525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1191/2382] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:20:48 2022] PUT /identity/v3/projects/62556a3d77ac46cd86af53cb22884a4c/users/09d3e38998394634a141e5d34bf21dda/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:20:48.773260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289e76d-72a7-4345-a5ef-5ea68a107103 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.773605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289e76d-72a7-4345-a5ef-5ea68a107103 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.773880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289e76d-72a7-4345-a5ef-5ea68a107103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.808476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d289e76d-72a7-4345-a5ef-5ea68a107103 None None] MFA Rules not processed for user `09d3e38998394634a141e5d34bf21dda`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:20:48.810438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d289e76d-72a7-4345-a5ef-5ea68a107103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:48.889692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1192/2383] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:20:48.903115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:48.904853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:48.962025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a63fd-2570-4398-b84a-eea542afc430 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:48.966873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:48.967495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:48.967782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:48.968054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:48.978263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:48.979701 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:48.989912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1192/2384] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:20:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:20:51.504673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:51.505234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:51.505234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:51.526612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] MFA Rules not processed for user `09d3e38998394634a141e5d34bf21dda`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:20:51.527839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:51.535126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1193/2385] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:20:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:20:58.175823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:58.177298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:58.236231 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160105d-40e8-4d6f-8e3a-41e0290402f3 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:58.239912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:58.240549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:58.240843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:58.241109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:58.242843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:58.315754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:58.317114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42649730-84f9-46d8-a02e-79ecdadab8ef service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:58.328534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1193/2386] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:20:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:20:58.379277 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:20:58.389543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc383d78-d188-4065-85f1-97d0870bb3df None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:20:58.393101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:20:58.393703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:20:58.393973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:20:58.394238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:20:58.396011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:20:58.428370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:20:58.428370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:20:58.434524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:20:58.446429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1194/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:20:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:08.074449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80edcb-f614-4362-a286-8ee6c85c6dfc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:08.075173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80edcb-f614-4362-a286-8ee6c85c6dfc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:08.075173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80edcb-f614-4362-a286-8ee6c85c6dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:08.094610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0e80edcb-f614-4362-a286-8ee6c85c6dfc None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:08.096001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0e80edcb-f614-4362-a286-8ee6c85c6dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:08.103676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:21:08.169320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1194/2388] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:21:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:08.216675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:08.218443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:08.307381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934516d-f1bf-4a91-aabe-16f1fd744341 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:08.316241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:08.316241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:08.316746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:08.317034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:08.333651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:08.335368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:08.352596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1195/2389] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:21:13.700182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d91434-1901-4740-ae40-a93dfa3157ff None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:13.700852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d91434-1901-4740-ae40-a93dfa3157ff None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:13.700852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d91434-1901-4740-ae40-a93dfa3157ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:13.727195 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2d91434-1901-4740-ae40-a93dfa3157ff None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:13.729819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2d91434-1901-4740-ae40-a93dfa3157ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:13.740268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1195/2390] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:21:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:13.755527 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:13.768455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38c687-2594-4781-8369-d10364b43dcf service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:13.775572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:13.776299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:13.777786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:13.778341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:13.787893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:13.789548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:13.800498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1196/2391] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 11:21:14.478180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a7b94-7abc-43b0-a087-3f057ead2303 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:14.478694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a7b94-7abc-43b0-a087-3f057ead2303 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:14.478972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a7b94-7abc-43b0-a087-3f057ead2303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:14.500651 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-547a7b94-7abc-43b0-a087-3f057ead2303 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:14.502181 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-547a7b94-7abc-43b0-a087-3f057ead2303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:14.524123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1196/2392] 10.222.0.16 () {58 vars in 967 bytes} [Thu Aug 18 11:21:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:14.570041 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:14.582920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56362efa-9fd7-4dbe-bbb4-d80ccb6f2066 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:14.594488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:14.595367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:14.595907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:14.596304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:14.607114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:14.608769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:14.618269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1197/2393] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:21:15.102437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f475-a69e-42ad-96c5-1638558c1510 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:15.103664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f475-a69e-42ad-96c5-1638558c1510 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:15.103664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f475-a69e-42ad-96c5-1638558c1510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:15.133048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6654f475-a69e-42ad-96c5-1638558c1510 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:15.134394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6654f475-a69e-42ad-96c5-1638558c1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:15.142831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1197/2394] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:21:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:21:15.159239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:15.171583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f667b-c14c-4d6b-9956-d1994946a2d3 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:15.177529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:15.178137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:15.178445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:15.178717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:15.188805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:15.190430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:15.200040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1198/2395] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:21:15.632907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:15.633211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:15.633463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:15.659085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:15.661516 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:15.671268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1198/2396] 10.222.0.16 () {58 vars in 967 bytes} [Thu Aug 18 11:21:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:15.688353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:15.699619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260cdac-4661-45cf-bf8a-f2f7716522c1 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:15.705772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:15.707312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:15.707312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:15.707312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:15.720411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:15.722345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:15.732980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1199/2397] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:21:17.821240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:17.823696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:17.843324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c296d7-ef77-4f30-9d18-bb566f74619c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:17.851896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:17.852747 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:17.854183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:17.854591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:17.857608 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:17.935996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:17.937636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6dc92-b2fc-4622-b806-dfa89d71169b service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:17.948387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1199/2398] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:18.164846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:18.269549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e65841-33bb-4147-80cd-e5d7060055c5 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:18.279934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:18.281053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:18.281400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:18.282173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] PATH_INFO: `/v3/users/09d3e38998394634a141e5d34bf21dda` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:18.283831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d3e38998394634a141e5d34bf21dda)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:18.287043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:18.347706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:21:18.358132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d3e38998394634a141e5d34bf21dda'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.370610 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d3e38998394634a141e5d34bf21dda'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.380583 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d3e38998394634a141e5d34bf21dda'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.381307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d3e38998394634a141e5d34bf21dda'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.382118 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d3e38998394634a141e5d34bf21dda'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.396533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1200/2399] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:21:18 2022] DELETE /identity/v3/users/09d3e38998394634a141e5d34bf21dda => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:21:18.401520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:18.401901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:18.402902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:18.423715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:18.425004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:18.490523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1200/2400] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:18.502349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:18.503797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:18.573043 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82804500-0ecb-4332-a05f-d74fdd08800f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:18.578090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:18.578706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:18.578984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:18.579255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:18.588845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:18.590890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:18.601062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1201/2401] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:21:18.781442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:18.783261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:18.797490 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9c99-4222-4819-9322-b4a5af94dbc6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:18.807205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:18.809592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:18.809955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:18.810268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] PATH_INFO: `/v3/projects/62556a3d77ac46cd86af53cb22884a4c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:18.811358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=62556a3d77ac46cd86af53cb22884a4c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:18.813339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:18.840266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:21:18.866309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62556a3d77ac46cd86af53cb22884a4c', 'request_id': 'req-60a26ae3-edfd-426b-b5a5-f435c00b5fac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.874555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62556a3d77ac46cd86af53cb22884a4c', 'request_id': 'req-60a26ae3-edfd-426b-b5a5-f435c00b5fac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.882180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62556a3d77ac46cd86af53cb22884a4c', 'request_id': 'req-60a26ae3-edfd-426b-b5a5-f435c00b5fac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.894542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] The token cache is being invalidate because project 62556a3d77ac46cd86af53cb22884a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:21:18.894940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.895669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60a26ae3-edfd-426b-b5a5-f435c00b5fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:21:18.897512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1201/2402] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:21:18 2022] DELETE /identity/v3/projects/62556a3d77ac46cd86af53cb22884a4c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:21:23.414065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:23.415476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:23.427463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849d22-5f06-4dfd-81f9-6873d368b083 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:23.483923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:23.484437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:23.484711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:23.484977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:23.486686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:23.496502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:23.498264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8763e-c9a5-4d1c-ac1c-718bc9edce20 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:23.506272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1202/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:23.641449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c02ef-268e-4805-939c-620066065fbd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:23.641881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c02ef-268e-4805-939c-620066065fbd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:23.641983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c02ef-268e-4805-939c-620066065fbd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:23.643968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1202/2404] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:21:23.654154 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338ffaa-43f9-4a41-8625-4d4028b1fb2f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:23.654521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338ffaa-43f9-4a41-8625-4d4028b1fb2f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:23.654725 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338ffaa-43f9-4a41-8625-4d4028b1fb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:23.676706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6338ffaa-43f9-4a41-8625-4d4028b1fb2f None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:23.678010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6338ffaa-43f9-4a41-8625-4d4028b1fb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:23.751693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:21:23.796404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1203/2405] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:21:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:23.808167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c02ef-268e-4805-939c-620066065fbd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:23.810449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-090c02ef-268e-4805-939c-620066065fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:23.874498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c02ef-268e-4805-939c-620066065fbd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:23.878048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5db5-7bf3-4623-9de1-b62fb4a71e88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:23.878642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5db5-7bf3-4623-9de1-b62fb4a71e88 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:23.878916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5db5-7bf3-4623-9de1-b62fb4a71e88 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:23.879185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5db5-7bf3-4623-9de1-b62fb4a71e88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:23.890853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5db5-7bf3-4623-9de1-b62fb4a71e88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:23.892344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5db5-7bf3-4623-9de1-b62fb4a71e88 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:23.904232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1203/2406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:23.978541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2dfe7-aad1-4dcd-b40f-c46906d6eb26 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:23.978896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2dfe7-aad1-4dcd-b40f-c46906d6eb26 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:23.979221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2dfe7-aad1-4dcd-b40f-c46906d6eb26 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:23.981120 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1204/2407] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:21:23.986769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892682d5-b102-4b96-8d27-b1c2dfb74431 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:23.987127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892682d5-b102-4b96-8d27-b1c2dfb74431 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:23.987435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892682d5-b102-4b96-8d27-b1c2dfb74431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:24.008000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-892682d5-b102-4b96-8d27-b1c2dfb74431 None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:24.009185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-892682d5-b102-4b96-8d27-b1c2dfb74431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:24.017559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1204/2408] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:21:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:24.032600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2dfe7-aad1-4dcd-b40f-c46906d6eb26 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:24.034875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-92c2dfe7-aad1-4dcd-b40f-c46906d6eb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:24.049464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2dfe7-aad1-4dcd-b40f-c46906d6eb26 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:24.053084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8dbbe-91d7-4c40-9a9d-afdb5394e3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:24.053683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8dbbe-91d7-4c40-9a9d-afdb5394e3bc service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:24.053962 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8dbbe-91d7-4c40-9a9d-afdb5394e3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:24.054231 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8dbbe-91d7-4c40-9a9d-afdb5394e3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:24.061826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8dbbe-91d7-4c40-9a9d-afdb5394e3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:24.063285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8dbbe-91d7-4c40-9a9d-afdb5394e3bc service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:24.071121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1205/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:24.160000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cf2e6-0e84-402a-9f9d-7956577f8cb9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:24.160439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cf2e6-0e84-402a-9f9d-7956577f8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:24.160825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cf2e6-0e84-402a-9f9d-7956577f8cb9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:24.162614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1205/2410] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:21:24.167946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:24.168344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:24.168643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:24.189319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:24.190551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:24.200576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1206/2411] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:21:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:24.211351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cf2e6-0e84-402a-9f9d-7956577f8cb9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:24.213074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-233cf2e6-0e84-402a-9f9d-7956577f8cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:24.226699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cf2e6-0e84-402a-9f9d-7956577f8cb9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:24.230271 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:24.231209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:24.231491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:24.231761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:24.241991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:24.243405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:24.255456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1206/2412] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:24.322571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:24.324673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:24.400879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb161-80ee-436d-8e5c-6f3f658bb67d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:24.406772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:24.407400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:24.407680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:24.407951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:24.410216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:24.424853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:24.426479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1c509-2091-4760-b5bb-c69922e36635 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:24.439248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1207/2413] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:25.883631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:25.893156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e6037-49d6-4954-8379-1b828957fc3c service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:25.897179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:25.897857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:25.898407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:25.898680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:25.900511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:25.911374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:25.912840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:25.920818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1207/2414] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:26.817656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:26.818150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:26.818242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:26.838003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:26.839220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:26.897831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1208/2415] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:21:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:26.911159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:26.912837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:26.982967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae9705-05bf-441e-9492-2d01ad525246 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:26.988050 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:26.988694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:26.988972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:26.989268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:26.997940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:26.999562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:27.008148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1208/2416] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:21:36.891057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=70 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:21:37.004799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:21:55.961798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:55.963275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:55.980986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0efc5-0ec6-4163-a5fa-a9d79742f0da None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:55.987187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:55.987810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:55.988082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:55.988386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:55.990327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:56.003200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:56.004845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dabbb3-785e-4af6-bb5b-0d816d05ed84 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:56.017754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1209/2417] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:56.530840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:56.543497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dfd45-566c-4498-a372-f29ca6cf8dc2 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:56.549627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:56.550405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:56.550842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:56.551221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:56.553508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:56.620213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:56.622361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:56.633731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1209/2418] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:21:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:21:58.928924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:58.929430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:58.929556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:58.949366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:21:58.950599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:21:58.957794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1210/2419] 10.222.0.16 () {58 vars in 967 bytes} [Thu Aug 18 11:21:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:21:58.969682 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:21:58.981208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20731d27-461d-456b-9cc5-7e10b98f4bcd service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:21:58.986207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:21:58.986799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:21:58.987073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:21:58.987342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:21:58.996400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:21:58.998054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:21:59.006548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1210/2420] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:21:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:22:08.943987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=9 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:22:09.003188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=34 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:22:28.259138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:22:28.268789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d6d01-dfc1-4889-a0db-42daa586a239 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:22:28.272477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:22:28.273098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:22:28.273374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:22:28.273643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:22:28.275674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:22:28.331017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:22:28.332502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:22:28.341305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:22:28.342949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:22:28.353180 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1211/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:22:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:22:28.353418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b13b9-a590-48bf-aeb6-7d1e6c514e90 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:22:28.359726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf9c6e-01fa-40e5-bed3-ab74cf88a757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:22:28.360369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9c6e-01fa-40e5-bed3-ab74cf88a757 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:22:28.360650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9c6e-01fa-40e5-bed3-ab74cf88a757 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:22:28.360918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9c6e-01fa-40e5-bed3-ab74cf88a757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:22:28.371999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf9c6e-01fa-40e5-bed3-ab74cf88a757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:22:28.374045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf9c6e-01fa-40e5-bed3-ab74cf88a757 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:22:28.385372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1211/2422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:22:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:22:29.867197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:22:29.890754 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5c277-d930-4b06-a6c3-c0b2e339e2ec service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:22:29.899650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:22:29.900919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:22:29.901995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:22:29.903466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:22:29.905750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:22:29.919721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:22:29.921598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3200-9e7e-4cc5-9ad6-cf400450e98a service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:22:29.948628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1212/2423] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:22:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:22:29.979472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:22:29.979952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:22:29.980940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:22:30.001916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] MFA Rules not processed for user `976558170ae64a9cbbd6930e7c3b048a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:22:30.003454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:22:30.011407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1212/2424] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:22:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:22:30.907596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:22:30.908516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:22:30.908987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:22:30.929692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] MFA Rules not processed for user `7ea15f54d31149ea948650cdd363d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:22:30.931162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:22:30.943366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1213/2425] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:22:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:22:37.969408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:22:37.983950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c65e0e-583e-4c86-b6ad-f01169a98efe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:22:37.990105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:22:37.990867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:22:37.994297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:22:37.994753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:22:37.997158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:22:38.010666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:22:38.012922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9ca9d-3fe3-43cb-b45b-d5bf4f6b2036 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:22:38.024703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1213/2426] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:22:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:22:40.923419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=37 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:22:48.019259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=15 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:23:06.662912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:23:06.676480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d945ac-8a14-415b-993a-b34b8a2cbe9c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:23:06.681842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:23:06.682439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:23:06.682718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:23:06.682989 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:23:06.685362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:23:06.744507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:23:06.746888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2cec4-adee-44a5-bd38-a73955772544 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:23:06.755896 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1214/2427] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:23:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:23:16.751696 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=28 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:23:18.630209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:23:18.630734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:23:18.630734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:23:18.649491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] MFA Rules not processed for user `d045f032dbfb450aa9c087ada1637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:23:18.650688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:23:18.658590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1214/2428] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:23:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:23:28.644762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:23:31.319040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8c80-e0ed-453a-a0e4-a93ad37a3e39 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:23:31.319531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8c80-e0ed-453a-a0e4-a93ad37a3e39 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:23:31.319626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8c80-e0ed-453a-a0e4-a93ad37a3e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:23:31.338172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f70d8c80-e0ed-453a-a0e4-a93ad37a3e39 None None] MFA Rules not processed for user `dae3738120794f9fa52c8054e3c462ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:23:31.339443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f70d8c80-e0ed-453a-a0e4-a93ad37a3e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:23:31.394366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1215/2429] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:23:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:23:41.387679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=25 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:23:59.678693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:23:59.749057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e68200-5357-4f40-abf8-988f03ab44be None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:23:59.755016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:23:59.755656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:23:59.755963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:23:59.756265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:23:59.758079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:23:59.835690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:23:59.837399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:23:59.850188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1215/2430] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:23:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:05.565835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:05.566250 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:05.566515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:05.598186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:24:05.599421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:05.607841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1216/2431] 10.222.0.16 () {58 vars in 967 bytes} [Thu Aug 18 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:24:05.675941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:05.706838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753e634-e773-4d8c-b759-626ee708dc8c service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:05.713621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:05.714336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:05.714708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:05.715390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:05.725265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:05.727178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:05.737628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1216/2432] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:24:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:24:06.038293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:06.040104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:06.089917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c07f0-3acc-450a-8ccd-c65e9f95bc1b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:06.094507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:06.095374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:06.095837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:06.096318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:06.098444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:06.112135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:06.117218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:06.118908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:06.122098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82863017-bb65-4114-a436-2acfe5e977a9 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:06.126654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:06.127254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:06.127523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:06.127960 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:06.129812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:06.130646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1217/2433] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:06.244099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:06.245587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:06.250957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:06.278143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1217/2434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:09.264489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:09.278203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87739e0-4ef5-45ca-9036-df6f8abddcb9 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:09.284520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:09.285199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:09.285603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:09.286122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:09.288067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:09.300684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:09.302314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:09.314368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1218/2435] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:13.368792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:13.382093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06440a97-90e6-4591-8dec-f2cf37e1f818 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:13.388841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:13.389263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:13.389593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:13.389869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:13.391705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:13.405273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:13.406954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:13.422323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1218/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:13.447191 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:13.449031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:13.464394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb3fa-7114-4c9d-a89a-42f9da126c6f service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:13.470632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:13.471396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:13.471736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:13.472076 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] PATH_INFO: `/v3/users/ebfd685af8cd428bbed9a35ef0325f8b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:13.473361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebfd685af8cd428bbed9a35ef0325f8b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:13.475354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:13.531693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebfd685af8cd428bbed9a35ef0325f8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.544271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebfd685af8cd428bbed9a35ef0325f8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.552030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebfd685af8cd428bbed9a35ef0325f8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.552682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebfd685af8cd428bbed9a35ef0325f8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.553373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebfd685af8cd428bbed9a35ef0325f8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.566005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1219/2437] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:24:13 2022] DELETE /identity/v3/users/ebfd685af8cd428bbed9a35ef0325f8b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:24:13.810205 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:13.821414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db183637-df82-4d75-b3f8-e2670a7422b8 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:13.881766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:13.882405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:13.882699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:13.882976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] PATH_INFO: `/v3/projects/c14bc6c22cf74ac7b59d0c55e82c0829` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:13.884139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14bc6c22cf74ac7b59d0c55e82c0829)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:13.886088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:13.937923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14bc6c22cf74ac7b59d0c55e82c0829', 'request_id': 'req-c510c077-e62a-4ee2-ad76-be5c0426f955'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.945593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14bc6c22cf74ac7b59d0c55e82c0829', 'request_id': 'req-c510c077-e62a-4ee2-ad76-be5c0426f955'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.952611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14bc6c22cf74ac7b59d0c55e82c0829', 'request_id': 'req-c510c077-e62a-4ee2-ad76-be5c0426f955'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.965270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] The token cache is being invalidate because project c14bc6c22cf74ac7b59d0c55e82c0829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:24:13.965549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.966367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:13.970956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1219/2438] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:24:13 2022] DELETE /identity/v3/projects/c14bc6c22cf74ac7b59d0c55e82c0829 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:24:13.973023 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:13.974478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:13.991367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18294d28-5197-4cff-a906-5d294e35b7c6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:14.089047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:14.089740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:14.090809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:14.091089 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] PATH_INFO: `/v3/users/6e8eb83d69f64a3cb6cf72c253fd97a2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:14.092392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8eb83d69f64a3cb6cf72c253fd97a2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:14.094347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:14.111071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:14.181069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8eb83d69f64a3cb6cf72c253fd97a2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.201984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8eb83d69f64a3cb6cf72c253fd97a2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.209660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8eb83d69f64a3cb6cf72c253fd97a2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.211024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8eb83d69f64a3cb6cf72c253fd97a2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.211704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8eb83d69f64a3cb6cf72c253fd97a2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.227421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1220/2439] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:24:13 2022] DELETE /identity/v3/users/6e8eb83d69f64a3cb6cf72c253fd97a2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:24:14.397055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:14.398691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:14.413445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510c077-e62a-4ee2-ad76-be5c0426f955 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:14.476482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:14.477008 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:14.477282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:14.477551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] PATH_INFO: `/v3/projects/26b16514d9854dfeba0ba85334246d05` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:14.478652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b16514d9854dfeba0ba85334246d05)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:14.480847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:14.488931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:14.532839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b16514d9854dfeba0ba85334246d05', 'request_id': 'req-c7b58b71-624d-495a-88a0-9caeb066734e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.541012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b16514d9854dfeba0ba85334246d05', 'request_id': 'req-c7b58b71-624d-495a-88a0-9caeb066734e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.548512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b16514d9854dfeba0ba85334246d05', 'request_id': 'req-c7b58b71-624d-495a-88a0-9caeb066734e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.560845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] The token cache is being invalidate because project 26b16514d9854dfeba0ba85334246d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:24:14.561162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.561971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:14.563550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1220/2440] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:24:14 2022] DELETE /identity/v3/projects/26b16514d9854dfeba0ba85334246d05 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:24:15.144495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:15.145939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:15.204496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aca78-913c-427c-aab2-80acff7406d6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:15.253133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:15.253733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:15.254011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:15.254280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:15.256001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:15.315738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:15.318108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:15.320266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:15.331750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1221/2441] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:18.724254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:18.725812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:18.739713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b58b71-624d-495a-88a0-9caeb066734e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:18.744026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:18.744717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:18.744997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:18.745274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] PATH_INFO: `/v3/users/976558170ae64a9cbbd6930e7c3b048a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:18.747043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] RBAC: Authorizing `identity:delete_user(user_id=976558170ae64a9cbbd6930e7c3b048a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:18.749110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:18.797257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '976558170ae64a9cbbd6930e7c3b048a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:18.797903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '976558170ae64a9cbbd6930e7c3b048a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:18.798505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '976558170ae64a9cbbd6930e7c3b048a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:18.810817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '976558170ae64a9cbbd6930e7c3b048a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:18.818766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a601fcd9-ccce-43fc-bb4f-b95f8d57455a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '976558170ae64a9cbbd6930e7c3b048a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:18.837572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1221/2442] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:24:18 2022] DELETE /identity/v3/users/976558170ae64a9cbbd6930e7c3b048a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:24:19.010256 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.020713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff7562-dbb5-4dd7-9da5-ff913cf44475 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.095436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.096419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.096697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.096968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] PATH_INFO: `/v3/projects/aab35f7119e44d3c8da92076c7e823db` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.098106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab35f7119e44d3c8da92076c7e823db)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.100030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.150868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab35f7119e44d3c8da92076c7e823db', 'request_id': 'req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:19.158058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab35f7119e44d3c8da92076c7e823db', 'request_id': 'req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:19.166237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab35f7119e44d3c8da92076c7e823db', 'request_id': 'req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:19.177917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] The token cache is being invalidate because project aab35f7119e44d3c8da92076c7e823db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:24:19.178202 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:19.178834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-16874d5e-c3ab-4c1a-ac35-b5c5c288c9d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:24:19.180359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1222/2443] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:24:19 2022] DELETE /identity/v3/projects/aab35f7119e44d3c8da92076c7e823db => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:24:19.186322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.186612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.186888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.244900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:24:19.246805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:19.280823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:19.308775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1222/2444] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:24:19.316653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.316940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.317215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.336689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:24:19.337910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:19.346432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1223/2445] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:24:19.350582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.361372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5cf4c-600c-48cb-aa60-2baddbf92ecd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.365881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.366469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.366740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.367006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.367643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:24:19.367941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.369579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.381283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1223/2446] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:24:19.385855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.395013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8df5d-5a46-4b85-a2a2-41e21d43baf3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.399801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.400460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.400743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.401014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.401625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.404388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.433108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1224/2447] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:24:19 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:24:19.437635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.447964 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda7889-3439-4705-9c32-d168bf44116c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.508660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.509253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.509530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.509797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.510417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.512467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.534318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1224/2448] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:24:19 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:24:19.538530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.549585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14453f-9426-4707-b1f2-b572030d088b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.554187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.554777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.555053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.555320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.556212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.557820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.567665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1225/2449] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:24:19.571623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.581818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a223b4-ea7b-4a3c-998b-755befd860b7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.586431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2182d6-595e-442b-840d-a7100bec169f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.587027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2182d6-595e-442b-840d-a7100bec169f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.587307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2182d6-595e-442b-840d-a7100bec169f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.587579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2182d6-595e-442b-840d-a7100bec169f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.588236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2182d6-595e-442b-840d-a7100bec169f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.590340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2182d6-595e-442b-840d-a7100bec169f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.600741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1225/2450] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:24:19.605232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.615854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadd949-e2c6-4ac3-b907-112faa41fcb3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.620573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.621253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.621530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.621873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] PATH_INFO: `/v3/projects/28a8db4d53754a09aad85d3b756fed10/users/c59d7dff52ce4e1f83039f3fe2288e6a/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.637248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a8db4d53754a09aad85d3b756fed10, user_id=c59d7dff52ce4e1f83039f3fe2288e6a, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.640002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.654056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1226/2451] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:24:19 2022] PUT /identity/v3/projects/28a8db4d53754a09aad85d3b756fed10/users/c59d7dff52ce4e1f83039f3fe2288e6a/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:24:19.659222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf7b30-ae72-417d-a314-16d839163ad0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.659537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf7b30-ae72-417d-a314-16d839163ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.659842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf7b30-ae72-417d-a314-16d839163ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.678838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4dcf7b30-ae72-417d-a314-16d839163ad0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:24:19.680060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4dcf7b30-ae72-417d-a314-16d839163ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:19.739146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1226/2452] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:24:19.749548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:19.751297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:19.776686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:19.831294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2bb41-82af-412b-aa05-28f7d85badc8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:19.834885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:19.835482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:19.835769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:19.836035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:19.845614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:19.847085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:19.857013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1227/2453] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:23.814184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06db7c8-ed35-4256-8e87-2059d715e903 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:23.814705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06db7c8-ed35-4256-8e87-2059d715e903 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:23.814705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06db7c8-ed35-4256-8e87-2059d715e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:23.845555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b06db7c8-ed35-4256-8e87-2059d715e903 None None] MFA Rules not processed for user `c59d7dff52ce4e1f83039f3fe2288e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:24:23.846790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b06db7c8-ed35-4256-8e87-2059d715e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:23.881798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:23.916769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1227/2454] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:24:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:24:23.931959 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:24:23.933526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:24.055199 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b4317-538e-4dc2-9954-6a2ffd23c9b3 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:24:24.061722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fadf15-19d3-4fef-9cba-9c19d5dca421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:24:24.062336 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fadf15-19d3-4fef-9cba-9c19d5dca421 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:24.062613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fadf15-19d3-4fef-9cba-9c19d5dca421 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:24.062882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fadf15-19d3-4fef-9cba-9c19d5dca421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:24.071447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fadf15-19d3-4fef-9cba-9c19d5dca421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:24:24.073135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fadf15-19d3-4fef-9cba-9c19d5dca421 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:24:24.083925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1228/2455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:24:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:24:25.442828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c80c9b-944a-4d25-9c0f-6d9d07f33f4a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:24:25.443538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c80c9b-944a-4d25-9c0f-6d9d07f33f4a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:24:25.443538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c80c9b-944a-4d25-9c0f-6d9d07f33f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:24:25.463981 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-11c80c9b-944a-4d25-9c0f-6d9d07f33f4a None None] MFA Rules not processed for user `c59d7dff52ce4e1f83039f3fe2288e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:24:25.466096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11c80c9b-944a-4d25-9c0f-6d9d07f33f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:24:25.473799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1228/2456] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:24:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:24:34.079248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=64 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:24:35.458198 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=15 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:25:19.263547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:19.264143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:19.264143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:19.284514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] MFA Rules not processed for user `7ea15f54d31149ea948650cdd363d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:25:19.285741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:19.341561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1229/2457] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:25:27.373818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:27.375012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:27.375174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:27.394459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] MFA Rules not processed for user `d045f032dbfb450aa9c087ada1637ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:25:27.395741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:27.453725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1229/2458] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:25:27.782752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:25:27.784172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:27.795840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecfd6a-e4b5-4aee-b6cd-f2bfc465f3ec None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:25:27.800350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:25:27.800949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:27.801224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:27.801491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:27.803183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:27.813971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:25:27.815394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380f0b0-8dca-4419-afe0-caec6d0d9618 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:25:27.823789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1230/2459] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:25:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:25:37.446060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:25:37.819888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=31 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:25:39.542455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:25:39.554485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63f45d-936f-4f2a-a19e-43b3a9f57be5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:25:39.558462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:25:39.559152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:39.559499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:39.559863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:39.561773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:39.630795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:25:39.632270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:25:39.642826 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1230/2460] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:25:49.638206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=30 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:25:53.621940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f429f5-861d-4799-a845-71bc38646adc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:53.622542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f429f5-861d-4799-a845-71bc38646adc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:53.622669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f429f5-861d-4799-a845-71bc38646adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:53.642747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-65f429f5-861d-4799-a845-71bc38646adc None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:25:53.643985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-65f429f5-861d-4799-a845-71bc38646adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:53.700404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1231/2461] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:25:53.748502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:25:53.749937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:53.824775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5ff6f-f617-4829-88f8-4332b34f47e6 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:25:53.829957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:25:53.830560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:53.830839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:53.831111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:53.839207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:25:53.840732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:25:53.850562 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1231/2462] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:25:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:25:58.399442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f429f5-861d-4799-a845-71bc38646adc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:25:58.414399 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f429f5-861d-4799-a845-71bc38646adc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:25:58.418784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:25:58.419722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:58.420232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:58.420685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:58.423172 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:58.439192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:25:58.440803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03041fd-1ece-4654-a826-57cc06062a8e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:25:58.462375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1232/2463] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:25:59.746681 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:25:59.760252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b439a36-a25a-4a1d-ab74-1b703df41465 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:25:59.761578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:25:59.762443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:25:59.762729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:25:59.763002 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:25:59.764814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:25:59.784504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:25:59.787479 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2030dc2-0d9f-4527-b518-2293a75567bb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:25:59.808668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1232/2464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:25:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:26:08.154828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46ae1-64ba-4400-94f0-e432865d2c70 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.155643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46ae1-64ba-4400-94f0-e432865d2c70 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.155786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46ae1-64ba-4400-94f0-e432865d2c70 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.157104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1233/2465] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:26:08.163631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccb031-a809-40c3-a0e5-72349e1d2f16 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.164129 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccb031-a809-40c3-a0e5-72349e1d2f16 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.164379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccb031-a809-40c3-a0e5-72349e1d2f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.205272 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d8ccb031-a809-40c3-a0e5-72349e1d2f16 None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:26:08.206520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d8ccb031-a809-40c3-a0e5-72349e1d2f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:08.216307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1233/2466] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:26:08.227955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46ae1-64ba-4400-94f0-e432865d2c70 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:08.229718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8fe46ae1-64ba-4400-94f0-e432865d2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:08.303361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46ae1-64ba-4400-94f0-e432865d2c70 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:08.306964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f0008-85c4-4deb-99eb-201a69bff8bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:08.307573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f0008-85c4-4deb-99eb-201a69bff8bc service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.307852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f0008-85c4-4deb-99eb-201a69bff8bc service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.308126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f0008-85c4-4deb-99eb-201a69bff8bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.318741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f0008-85c4-4deb-99eb-201a69bff8bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:08.320219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f0008-85c4-4deb-99eb-201a69bff8bc service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:08.331767 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1234/2467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:26:08.404645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6a78a-66ea-45a3-bb9e-3004013e41d3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.404931 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6a78a-66ea-45a3-bb9e-3004013e41d3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.405249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6a78a-66ea-45a3-bb9e-3004013e41d3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.406864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1234/2468] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:26:08.412568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75ff2d-e55c-4117-b0ef-dd5a58987f17 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.412858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75ff2d-e55c-4117-b0ef-dd5a58987f17 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.413170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75ff2d-e55c-4117-b0ef-dd5a58987f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.433580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7e75ff2d-e55c-4117-b0ef-dd5a58987f17 None None] MFA Rules not processed for user `a09c08eca5be4a1b8977ff91bb468358`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:26:08.434820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7e75ff2d-e55c-4117-b0ef-dd5a58987f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:08.441591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1235/2469] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:26:08.453594 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6a78a-66ea-45a3-bb9e-3004013e41d3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:08.465573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6a78a-66ea-45a3-bb9e-3004013e41d3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:08.468997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9596e-a3f5-4fc0-bbbd-648eb967eaba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:08.469582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9596e-a3f5-4fc0-bbbd-648eb967eaba service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.470498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9596e-a3f5-4fc0-bbbd-648eb967eaba service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.470942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9596e-a3f5-4fc0-bbbd-648eb967eaba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.478676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9596e-a3f5-4fc0-bbbd-648eb967eaba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:08.480098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9596e-a3f5-4fc0-bbbd-648eb967eaba service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:08.488895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1235/2470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:26:08.575632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdbfc6-ecf3-46c7-996f-ea079f92ecbc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.575919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdbfc6-ecf3-46c7-996f-ea079f92ecbc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.576215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdbfc6-ecf3-46c7-996f-ea079f92ecbc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.577590 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1236/2471] 10.222.0.16 () {54 vars in 921 bytes} [Thu Aug 18 11:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 11:26:08.583895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb365995-35a3-4e59-aea9-169fe4004603 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.584208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb365995-35a3-4e59-aea9-169fe4004603 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.584514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb365995-35a3-4e59-aea9-169fe4004603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.604721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fb365995-35a3-4e59-aea9-169fe4004603 None None] MFA Rules not processed for user `1f43ff51c5f84d34978c46bce6db67ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:26:08.605951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fb365995-35a3-4e59-aea9-169fe4004603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:08.616116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1236/2472] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Aug 18 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:26:08.627221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdbfc6-ecf3-46c7-996f-ea079f92ecbc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:08.638349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdbfc6-ecf3-46c7-996f-ea079f92ecbc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:08.641971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:08.642653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:08.642936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:08.643211 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:08.653659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:08.655082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:08.667580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1237/2473] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:26:10.853505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:10.854056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:10.854056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:10.873861 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:26:10.875079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:10.883156 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1237/2474] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:26:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:26:10.895856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:10.908308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bee72-4e39-45ea-b71e-e9b4306748d7 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:10.913420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:10.914135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:10.914428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:10.914702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:10.922996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:10.924824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:10.932877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1238/2475] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:26:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:26:17.699496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:17.700940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:17.716838 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a9a81-f9f7-41c4-90ce-e999878236e4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:17.723296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:17.723915 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:17.724213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:17.724501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:17.726279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:17.816348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:17.817947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0534-cefe-4ed5-9152-8864cd6eaf67 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:17.835940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:26:17.840220 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1238/2476] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:26:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:26:20.929647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=72 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:26:27.703314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:27.716243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d8de8-6a92-452e-8b57-373c1572abb4 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:27.720485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:27.721127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:27.721403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:27.721671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:27.723454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:27.736386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:27.738191 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:27.747294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1239/2477] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:26:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:26:35.439492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:35.440539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:35.440539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:35.497949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:26:35.499220 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:26:35.521390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1239/2478] 10.222.0.16 () {58 vars in 967 bytes} [Thu Aug 18 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:26:35.536319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:26:35.562602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae0770-60f2-42ec-a1bf-30801a4323c3 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:26:35.567722 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:26:35.568350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:26:35.568627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:26:35.568897 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:26:35.577428 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:26:35.579534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:26:35.587791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1240/2479] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:26:45.516093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=18 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:26:45.584664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=16 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:27:45.723209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:45.740534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849fb69-3948-440e-b0a3-4fc78f249b28 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:45.747107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:45.748567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:45.748865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:45.749145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:45.751244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:45.767255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:45.769157 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:45.795131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1240/2480] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:27:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:27:50.677931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:50.687468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad68d-69e4-4403-b56d-63fb3c0a608f service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:50.691796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:50.692430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:50.692709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:50.692980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:50.694690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:50.706504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:50.707915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:50.717587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1241/2481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:27:54.093307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:54.096152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:54.120489 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9b8b-66a4-410d-a901-82415488a75f service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:54.129432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:54.130055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.130330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.130600 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] PATH_INFO: `/v3/users/d045f032dbfb450aa9c087ada1637ee0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.147565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d045f032dbfb450aa9c087ada1637ee0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:54.151377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:54.211087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd045f032dbfb450aa9c087ada1637ee0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.211650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd045f032dbfb450aa9c087ada1637ee0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.212400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd045f032dbfb450aa9c087ada1637ee0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.226354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd045f032dbfb450aa9c087ada1637ee0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.234360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bec7359a-ff97-49a1-9c47-0d64cbadf123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd045f032dbfb450aa9c087ada1637ee0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.247784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1241/2482] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:27:54 2022] DELETE /identity/v3/users/d045f032dbfb450aa9c087ada1637ee0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:27:54.439486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:54.450026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade644a6-f9cb-43db-84c0-cf7db4519b55 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:54.516404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:54.517042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.517888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.518352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] PATH_INFO: `/v3/projects/129f3d9f33ad4b209039244eb09599e9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.530871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=129f3d9f33ad4b209039244eb09599e9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:54.533924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:54.584638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129f3d9f33ad4b209039244eb09599e9', 'request_id': 'req-4221c35a-36de-4f09-9fbd-9858a9da73fc'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.590270 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129f3d9f33ad4b209039244eb09599e9', 'request_id': 'req-4221c35a-36de-4f09-9fbd-9858a9da73fc'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.600158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129f3d9f33ad4b209039244eb09599e9', 'request_id': 'req-4221c35a-36de-4f09-9fbd-9858a9da73fc'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.612647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] The token cache is being invalidate because project 129f3d9f33ad4b209039244eb09599e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:27:54.612937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.613588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4221c35a-36de-4f09-9fbd-9858a9da73fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:27:54.615224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1242/2483] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:27:54 2022] DELETE /identity/v3/projects/129f3d9f33ad4b209039244eb09599e9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:27:54.620226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.620502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.620778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.640447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:27:54.641662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:54.702933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1242/2484] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:27:54.712709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.713067 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.713389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.734968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:27:54.736111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:54.744543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1243/2485] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:27:54.748879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:54.758975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de2362-063a-464a-a2fb-64337178f5a1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:54.763821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:54.764468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.764750 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.765025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.765695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:27:54.765986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:54.768133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:54.780523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1243/2486] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:27:54.786675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:54.796262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f9346-059f-4b94-8f7e-5ff9a2e9c6c6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:54.800929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:54.801530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.801806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.802075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.802706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:54.804646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:54.846900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1244/2487] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:27:54 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:27:54.851155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:54.861515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840a21-7dfd-4412-9ee7-c25f59b390e1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:54.905088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:27:54.933613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:54.934057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.934335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.934659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.935317 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:54.937474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:54.962268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1244/2488] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:27:54 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:27:54.966918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:54.979228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f2292-7ac7-4715-a004-42cb13c76975 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:54.984391 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:54.985233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:54.985461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:54.985736 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:54.986389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:54.988027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:54.999094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1245/2489] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:27:55.012521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:55.030894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0c10-27b0-44eb-8db3-a3350ae7762e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:55.036983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320062a-9a6b-41aa-b812-0734496f949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:55.038036 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320062a-9a6b-41aa-b812-0734496f949c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:55.038546 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320062a-9a6b-41aa-b812-0734496f949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:55.039022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320062a-9a6b-41aa-b812-0734496f949c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:55.039699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320062a-9a6b-41aa-b812-0734496f949c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:55.041370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320062a-9a6b-41aa-b812-0734496f949c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:55.054090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1245/2490] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:27:55.057859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:55.074441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38617b-c6d6-4c12-9a76-0f241a8a193c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:55.079350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:55.080703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:55.081184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:55.081606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] PATH_INFO: `/v3/projects/45d34816b44d4cb69c37debadefb8bb5/users/d60c3aab508b4e2086406a41a61de15f/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:55.097407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d34816b44d4cb69c37debadefb8bb5, user_id=d60c3aab508b4e2086406a41a61de15f, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:55.100072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:55.103639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:27:55.117254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1246/2491] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:27:55 2022] PUT /identity/v3/projects/45d34816b44d4cb69c37debadefb8bb5/users/d60c3aab508b4e2086406a41a61de15f/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:27:55.129966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:55.130614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:55.130614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:55.170658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] MFA Rules not processed for user `d60c3aab508b4e2086406a41a61de15f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:27:55.171917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:55.246804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1246/2492] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:27:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:27:55.258827 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:27:55.260627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:27:55.346795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9e4b4-7029-4b3d-808a-1aff86cc4600 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:27:55.356229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892280d2-7eb8-4efa-ab5b-b3647e29fb22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:27:55.356863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892280d2-7eb8-4efa-ab5b-b3647e29fb22 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:27:55.357137 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892280d2-7eb8-4efa-ab5b-b3647e29fb22 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:27:55.357404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892280d2-7eb8-4efa-ab5b-b3647e29fb22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:27:55.366459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892280d2-7eb8-4efa-ab5b-b3647e29fb22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:27:55.368047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892280d2-7eb8-4efa-ab5b-b3647e29fb22 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:27:55.378585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1247/2493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:27:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:04.942011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:05.027596 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afa058-0187-47f5-bbbe-8a2c7e1bb4c8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:05.034824 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:05.035476 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.035758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.036030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] PATH_INFO: `/v3/users/d60c3aab508b4e2086406a41a61de15f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.037216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60c3aab508b4e2086406a41a61de15f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:05.039151 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:05.089708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:05.091672 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60c3aab508b4e2086406a41a61de15f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.092688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60c3aab508b4e2086406a41a61de15f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.093695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60c3aab508b4e2086406a41a61de15f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.111053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60c3aab508b4e2086406a41a61de15f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.123039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60c3aab508b4e2086406a41a61de15f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.146705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1247/2494] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:04 2022] DELETE /identity/v3/users/d60c3aab508b4e2086406a41a61de15f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:05.151193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.151715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.151715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.171614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:05.172877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.246060 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1248/2495] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:05.256711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:05.258139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.323233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e5b4f-86b0-46de-90d7-1e952e668db1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:05.326946 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:05.327554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.327835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.328110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.339260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:05.340842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:05.351804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1248/2496] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:05.640381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:05.642631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.657359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287458d7-52a6-43aa-8fb3-523dff6d5894 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:05.664361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:05.665201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.665655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.665931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] PATH_INFO: `/v3/projects/45d34816b44d4cb69c37debadefb8bb5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.667095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d34816b44d4cb69c37debadefb8bb5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:05.669182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:05.744946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d34816b44d4cb69c37debadefb8bb5', 'request_id': 'req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.755628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d34816b44d4cb69c37debadefb8bb5', 'request_id': 'req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.756302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:05.757996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.763839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d34816b44d4cb69c37debadefb8bb5', 'request_id': 'req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.773790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc70c4d-303b-4cdc-a004-29c526f8f514 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:05.778831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] The token cache is being invalidate because project 45d34816b44d4cb69c37debadefb8bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:05.779996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.779996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c1344a9-2bc9-4859-91f4-cf9d7c6e2b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:05.782187 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1249/2497] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:05 2022] DELETE /identity/v3/projects/45d34816b44d4cb69c37debadefb8bb5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:05.788218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60ea73-3040-401f-a904-a6cca8ace51d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.789105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60ea73-3040-401f-a904-a6cca8ace51d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.789105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60ea73-3040-401f-a904-a6cca8ace51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.805517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:05.819553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3f60ea73-3040-401f-a904-a6cca8ace51d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:05.821927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f60ea73-3040-401f-a904-a6cca8ace51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.831424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:05.832035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.832358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.832637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.834480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.889968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1250/2498] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:28:05.900105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.900419 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.900753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.917245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:05.918769 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:05.925210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:05.926445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:05.929890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1249/2499] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:28:05.942443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1251/2500] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:05.944039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:05.954269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb309d-53c2-423c-82af-46a58c515bb4 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:05.959898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:05.960534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:05.960807 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:05.961077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:05.961719 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:05.962015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:05.963601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:05.977374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1250/2501] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:05.982059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:05.992103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b0f98-c2f3-4182-ab86-81e21beaa5ab None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:05.998878 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:05.999515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.001354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.001354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.001354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.002963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.044237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1252/2502] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:05 2022] POST /identity/v3/projects => generated 357 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:06.047811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.055810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:06.066564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256fb4-ab11-478e-b49f-de54ce3499b0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.133878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.134497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.134777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.135049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.135680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.137683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.163721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1251/2503] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:06 2022] POST /identity/v3/users => generated 279 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:06.168042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.179182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1e02-d5e1-4632-bee4-326ee84bbf3a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.184296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.184925 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.185254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.185469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.186117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.187739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.198942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1253/2504] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:06.203209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.214378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11134ea-7938-4357-8982-d4217c9abff3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.219410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852966bb-c455-4f5c-ba88-21b9b1e92f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.220048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852966bb-c455-4f5c-ba88-21b9b1e92f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.220378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852966bb-c455-4f5c-ba88-21b9b1e92f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.220643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852966bb-c455-4f5c-ba88-21b9b1e92f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.221321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852966bb-c455-4f5c-ba88-21b9b1e92f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.222935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852966bb-c455-4f5c-ba88-21b9b1e92f28 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.235074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1252/2505] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:06.239491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.250312 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209761c4-eac8-41e0-a7f4-a2d45a50c398 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.256262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.257133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.257612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.258052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] PATH_INFO: `/v3/projects/23cbc7b2b92540a493a168e87b0e53bd/users/25c6dde2c21e455682620ee1982a6581/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.278375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23cbc7b2b92540a493a168e87b0e53bd, user_id=25c6dde2c21e455682620ee1982a6581, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.283495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.301246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1254/2506] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:06 2022] PUT /identity/v3/projects/23cbc7b2b92540a493a168e87b0e53bd/users/25c6dde2c21e455682620ee1982a6581/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:06.313221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.313669 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.314832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.344637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] MFA Rules not processed for user `25c6dde2c21e455682620ee1982a6581`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:06.354604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:06.437420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1253/2507] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:06.443585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.518491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0681e9-4af6-4f74-86e2-6268cd550587 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.523548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.524164 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.524472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.524750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.525380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.527594 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.566309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1255/2508] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:06 2022] POST /identity/v3/projects => generated 357 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:06.571815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.583983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92586-ae31-4064-9e61-4a796d128db2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.647448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.648119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.648475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.648638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.649264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.651282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.670790 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:06.681329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1254/2509] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:06 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:06.685382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.696972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a2014-3b4b-43c8-a04e-40f4b18cc262 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.701826 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.702445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.702721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.702996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.703642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.705322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.716596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1256/2510] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:06.721412 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.731690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3c15f-8dff-4ec0-9cf8-54148c07c1ff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.737962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.738788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.739006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.739277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] PATH_INFO: `/v3/projects/a71642b5faa7468da0b6a31e6b225efb/users/bf2ee6aa889941e2b1539ee45cd06849/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.763925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71642b5faa7468da0b6a31e6b225efb, user_id=bf2ee6aa889941e2b1539ee45cd06849, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.766650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.781673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1255/2511] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:06 2022] PUT /identity/v3/projects/a71642b5faa7468da0b6a31e6b225efb/users/bf2ee6aa889941e2b1539ee45cd06849/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:06.786553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.800090 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e781f9-936c-4231-8a2c-434760c10019 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.819848 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:06.886234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaa227-b91d-4926-b498-aa129d6fbfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.886894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaa227-b91d-4926-b498-aa129d6fbfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.887173 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaa227-b91d-4926-b498-aa129d6fbfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.887454 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaa227-b91d-4926-b498-aa129d6fbfa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.888091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaa227-b91d-4926-b498-aa129d6fbfa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.889772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaa227-b91d-4926-b498-aa129d6fbfa3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.903857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1257/2512] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:06.905075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:06.921857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fed90-e206-4d42-af47-f8e1f4140550 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:06.928210 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:06.929132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.929242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.929538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] PATH_INFO: `/v3/projects/a71642b5faa7468da0b6a31e6b225efb/users/bf2ee6aa889941e2b1539ee45cd06849/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:06.932467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71642b5faa7468da0b6a31e6b225efb, user_id=bf2ee6aa889941e2b1539ee45cd06849, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:06.935167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:06.949501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1256/2513] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:06 2022] PUT /identity/v3/projects/a71642b5faa7468da0b6a31e6b225efb/users/bf2ee6aa889941e2b1539ee45cd06849/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:06.966482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:06.966771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:06.968306 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:07.011513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] MFA Rules not processed for user `bf2ee6aa889941e2b1539ee45cd06849`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:07.012808 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:07.133294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1258/2514] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:07.149499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:07.151261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:07.252541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26275d-70fa-4bfb-8294-023132375405 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:07.256755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:07.257428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:07.257752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:07.258062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:07.270570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:07.272054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:07.283853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1257/2515] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:07.370014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:07.381298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23532cb-6ef7-40a8-9032-e0c21c1acb49 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:07.385316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:07.385921 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:07.386197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:07.386467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:07.460985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:07.462473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:07.473506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1259/2516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:09.490998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:09.492918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:09.583146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd19a-c181-48e9-aa6a-4e8da52d4429 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:09.589341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:09.590058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:09.590408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:09.590746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] PATH_INFO: `/v3/users/dae3738120794f9fa52c8054e3c462ed` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:09.591878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae3738120794f9fa52c8054e3c462ed)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:09.593886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:09.648629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae3738120794f9fa52c8054e3c462ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:09.649315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae3738120794f9fa52c8054e3c462ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:09.649995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae3738120794f9fa52c8054e3c462ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:09.651430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:09.666082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae3738120794f9fa52c8054e3c462ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:09.673518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f0750093-03d6-46f8-83fe-190d6c1579eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae3738120794f9fa52c8054e3c462ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:09.686302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1258/2517] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:09 2022] DELETE /identity/v3/users/dae3738120794f9fa52c8054e3c462ed => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:09.867847 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:09.888248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd95556-af55-4ae8-8411-0a75c3df38f8 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:09.934992 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:09.975161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:09.975161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:09.975161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:09.975161 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] PATH_INFO: `/v3/projects/be810e9efc4f4214b473f76da7e2714b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:09.975664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=be810e9efc4f4214b473f76da7e2714b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:09.975664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.034678 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be810e9efc4f4214b473f76da7e2714b', 'request_id': 'req-e04debde-aaf2-43d9-a752-ab7600798d6d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:10.042079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be810e9efc4f4214b473f76da7e2714b', 'request_id': 'req-e04debde-aaf2-43d9-a752-ab7600798d6d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:10.051540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be810e9efc4f4214b473f76da7e2714b', 'request_id': 'req-e04debde-aaf2-43d9-a752-ab7600798d6d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:10.069151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] The token cache is being invalidate because project be810e9efc4f4214b473f76da7e2714b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:10.069590 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:10.070763 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e04debde-aaf2-43d9-a752-ab7600798d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:10.071862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1260/2518] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:09 2022] DELETE /identity/v3/projects/be810e9efc4f4214b473f76da7e2714b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:10.077287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.077713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.078061 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.100999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:10.102423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:10.167123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1259/2519] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:10.176778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.177119 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.177460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.199987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:10.201862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:10.212321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1261/2520] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:10.218667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.230893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d330b-1f13-4974-af87-50b3c1bfdbd1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.244932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.246221 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.246656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.247081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.247955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:10.248462 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.250563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.269832 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1260/2521] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:10.271867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.282058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea48bd5-9878-4d86-8c1d-58859552a1a0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.287746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.290049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.290049 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.290403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.291764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.294195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.323745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1262/2522] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:10.327951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.339122 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545a1b-8012-4df1-a8ac-3d01175f6766 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.402590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.403444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.403900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.404397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.405431 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.409538 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.434780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1261/2523] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:10.439267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.450080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6201ad-4755-437e-b357-7384c7d7c063 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.455573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.456169 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.456474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.456742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.457389 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.458998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.469750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1263/2524] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:10.474795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.491526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406fbe2-f0a6-4cb6-b479-d4ca6d7bc852 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.499571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b814b91-42d0-4e9c-8730-628a1c6817be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.500217 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b814b91-42d0-4e9c-8730-628a1c6817be admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.500509 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b814b91-42d0-4e9c-8730-628a1c6817be admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.500785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b814b91-42d0-4e9c-8730-628a1c6817be admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.501542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b814b91-42d0-4e9c-8730-628a1c6817be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.504102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b814b91-42d0-4e9c-8730-628a1c6817be admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.516636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1262/2525] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:10.521828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.533285 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c0bea-749f-4524-9973-545a7134e019 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.538142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.538806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.539086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.539357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] PATH_INFO: `/v3/projects/1e3f936bbfdb4f498634836e269907d3/users/80ae2dc9271e4a5895b4b77d8c6f2ec0/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.559626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3f936bbfdb4f498634836e269907d3, user_id=80ae2dc9271e4a5895b4b77d8c6f2ec0, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.559626 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.576305 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1264/2526] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:10 2022] PUT /identity/v3/projects/1e3f936bbfdb4f498634836e269907d3/users/80ae2dc9271e4a5895b4b77d8c6f2ec0/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:10.588247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.588564 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.588839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.617752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] MFA Rules not processed for user `80ae2dc9271e4a5895b4b77d8c6f2ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:10.618997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:10.639498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:10.691868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1263/2527] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:10.696660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.764136 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433be34-a006-439e-a78e-a96ab1a05031 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.769870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.770474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.770749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.771015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.771628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.773561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.779170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:10.806314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1265/2528] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:10.812172 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.824041 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7531b-3c4b-40c9-96d5-9dc958713a46 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.885193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.885812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.886152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.886429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.887094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.889072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.911971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1264/2529] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:10 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:10.916397 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.927761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde69a91-2129-48f2-aaad-ce4fda9a8440 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.932602 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.933226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.933507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.933832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.934494 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.936125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:10.955879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1266/2530] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:10.957452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:10.968525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561f685-8604-4c3a-876a-b888985b566f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:10.973816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:10.974724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:10.975037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:10.975312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] PATH_INFO: `/v3/projects/fa74ed6be2884bf7a7f2401c66771823/users/73f9553b564d41d99a27eec49fa722df/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:10.993234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa74ed6be2884bf7a7f2401c66771823, user_id=73f9553b564d41d99a27eec49fa722df, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:10.995983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:11.012554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1265/2531] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:10 2022] PUT /identity/v3/projects/fa74ed6be2884bf7a7f2401c66771823/users/73f9553b564d41d99a27eec49fa722df/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:11.016882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:11.028710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc4dab-a6cf-44fc-92bb-f3a0534b8a33 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:11.100247 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c794b69-ad44-4c7d-9ae8-3f422129afb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:11.100872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c794b69-ad44-4c7d-9ae8-3f422129afb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:11.101146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c794b69-ad44-4c7d-9ae8-3f422129afb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:11.101412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c794b69-ad44-4c7d-9ae8-3f422129afb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:11.102051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c794b69-ad44-4c7d-9ae8-3f422129afb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:11.103649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c794b69-ad44-4c7d-9ae8-3f422129afb6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:11.122178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1267/2532] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:11.123477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:11.137207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdaf1e-4ac7-42b0-b455-51ce31c7248d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:11.143498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:11.144211 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:11.144581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:11.144804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] PATH_INFO: `/v3/projects/fa74ed6be2884bf7a7f2401c66771823/users/73f9553b564d41d99a27eec49fa722df/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:11.146805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa74ed6be2884bf7a7f2401c66771823, user_id=73f9553b564d41d99a27eec49fa722df, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:11.149645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:11.162152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1266/2533] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:11 2022] PUT /identity/v3/projects/fa74ed6be2884bf7a7f2401c66771823/users/73f9553b564d41d99a27eec49fa722df/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:11.175056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:11.175444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:11.175790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:11.206616 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] MFA Rules not processed for user `73f9553b564d41d99a27eec49fa722df`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:11.207792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:11.281701 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1268/2534] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:11.294053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:11.295932 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:11.368941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839c43c-df8f-4e32-b5f7-3899cb682cd8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:11.373111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:11.373802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:11.374456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:11.374693 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:11.385816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:11.387293 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:11.404741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1267/2535] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:11.518441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:11.538385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e7bed-3521-4bf5-825f-e822c3776bb2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:11.544690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:11.545480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:11.545965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:11.546534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:11.618303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:11.620877 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:11.622340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:11.644416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1269/2536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:14.002224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:14.003802 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:14.023158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:14.068457 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fdcba-2a7c-4acf-b0c2-c0b305c9f1fd service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:14.072141 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:14.072988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:14.073279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:14.073569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:14.075433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:14.159152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:14.160631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:14.179948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1268/2537] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:15.176422 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:15.178151 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:15.292167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be59e3-0988-47d7-9d84-2daa010f2742 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:15.310851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:15.311603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:15.311913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:15.312152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:15.314814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:15.335834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:15.338065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027e2c1-06a4-4b6f-90ee-b70bb587d487 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:15.351117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1270/2538] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:22.869604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:22.953144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af38f-a798-4b31-af22-99354627ca80 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:22.959918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:22.959918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:22.959918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:22.959918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] PATH_INFO: `/v3/users/80ae2dc9271e4a5895b4b77d8c6f2ec0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:22.961955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ae2dc9271e4a5895b4b77d8c6f2ec0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:22.964018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:23.022440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:23.025389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ae2dc9271e4a5895b4b77d8c6f2ec0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:23.026234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ae2dc9271e4a5895b4b77d8c6f2ec0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:23.026962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ae2dc9271e4a5895b4b77d8c6f2ec0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:23.041086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ae2dc9271e4a5895b4b77d8c6f2ec0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:23.049664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ae2dc9271e4a5895b4b77d8c6f2ec0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:23.062508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1269/2539] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:22 2022] DELETE /identity/v3/users/80ae2dc9271e4a5895b4b77d8c6f2ec0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:23.068005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:23.068597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:23.068714 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:23.092801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:23.094026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:23.162162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1271/2540] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:23.175495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:23.175495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:23.250029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c56aea-2a56-4443-aaa2-f6dd6fbfa990 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:23.254823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:23.255448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:23.255726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:23.256146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:23.267246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:23.268791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:23.279728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1270/2541] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:23.905036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:23.907133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:23.920260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d846a73-536b-429b-a2fd-cc6505c773df None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:23.925756 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:23.926507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:23.926787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:23.927059 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] PATH_INFO: `/v3/projects/1e3f936bbfdb4f498634836e269907d3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:23.928224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3f936bbfdb4f498634836e269907d3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:23.930292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:23.983942 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3f936bbfdb4f498634836e269907d3', 'request_id': 'req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:23.990693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3f936bbfdb4f498634836e269907d3', 'request_id': 'req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.000775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3f936bbfdb4f498634836e269907d3', 'request_id': 'req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.016734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] The token cache is being invalidate because project 1e3f936bbfdb4f498634836e269907d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:24.017006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.017685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.019207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1272/2542] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:23 2022] DELETE /identity/v3/projects/1e3f936bbfdb4f498634836e269907d3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:24.023886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:24.036022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a333d-5312-4755-8855-5a8d9b0cd877 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:24.101709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:24.102400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:24.102674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:24.102942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] PATH_INFO: `/v3/users/73f9553b564d41d99a27eec49fa722df` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:24.104087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f9553b564d41d99a27eec49fa722df)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:24.106103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:24.160279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f9553b564d41d99a27eec49fa722df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.161074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f9553b564d41d99a27eec49fa722df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.161740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f9553b564d41d99a27eec49fa722df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.175436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f9553b564d41d99a27eec49fa722df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.177451 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:24.186765 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4f9c7298-028d-463b-a7c7-0b327bd2233f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f9553b564d41d99a27eec49fa722df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.203782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1271/2543] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:24 2022] DELETE /identity/v3/users/73f9553b564d41d99a27eec49fa722df => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:24.519561 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:24.522315 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:24.540364 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:24.545353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b9c0-7269-4cab-b3ea-63b3c6f850cf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:24.650789 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:24.651479 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:24.651697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:24.652264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] PATH_INFO: `/v3/projects/fa74ed6be2884bf7a7f2401c66771823` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:24.654356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa74ed6be2884bf7a7f2401c66771823)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:24.656857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:24.741159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa74ed6be2884bf7a7f2401c66771823', 'request_id': 'req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.747452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa74ed6be2884bf7a7f2401c66771823', 'request_id': 'req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.758016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa74ed6be2884bf7a7f2401c66771823', 'request_id': 'req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.772021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] The token cache is being invalidate because project fa74ed6be2884bf7a7f2401c66771823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:24.772604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.773517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d96fbac7-3ab2-4151-b889-eb3c3cfa6d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:24.775416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1273/2544] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:24 2022] DELETE /identity/v3/projects/fa74ed6be2884bf7a7f2401c66771823 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:24.780423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:24.780710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:24.781015 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:24.806896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:24.808140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:24.923531 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1272/2545] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:24.931608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:24.931898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:24.932223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:24.956669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:24.959740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:24.972287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1274/2546] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:24.976948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:24.986969 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21a063-72ec-4053-9fd3-39cc315f7ae6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:24.993867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:24.994507 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:24.994788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:24.995062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:24.995761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:24.996059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:24.997792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.027796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1273/2547] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:28:25.030350 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.041830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e7987-e0ea-4380-acae-a5bb7abc5bd0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.047040 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.047964 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.048226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.048539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.049226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.051929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.092264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1275/2548] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:25 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:25.095166 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.106232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3eb77-1b7e-4ec9-9dce-8ebb4e8a2f01 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.201269 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.201883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.202164 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.202440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.203071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.205551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.233235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1274/2549] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:25 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:25.239016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.260372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205659cc-7bf5-4ef0-b98d-e9ef6c3a4561 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.269251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.270318 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.270825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.271326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.272563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.275786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.297486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1276/2550] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:25.302664 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.314587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de665aac-e5f3-4259-8d33-22c583265d97 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.324053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c591ae-d98d-4085-952a-2a0f810d5b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.324841 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c591ae-d98d-4085-952a-2a0f810d5b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.325159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c591ae-d98d-4085-952a-2a0f810d5b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.325539 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c591ae-d98d-4085-952a-2a0f810d5b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.326290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c591ae-d98d-4085-952a-2a0f810d5b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.328376 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c591ae-d98d-4085-952a-2a0f810d5b82 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.345154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1275/2551] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:25.349142 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.361152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1406b63-d9a2-4133-b527-406361434117 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.369033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.370418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.370872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.371334 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] PATH_INFO: `/v3/projects/c4bf3a1988c4435680aec058007a438f/users/157f4d1e32aa493d848cee54528e0f7b/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.394712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bf3a1988c4435680aec058007a438f, user_id=157f4d1e32aa493d848cee54528e0f7b, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.397525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.416773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1277/2552] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:25 2022] PUT /identity/v3/projects/c4bf3a1988c4435680aec058007a438f/users/157f4d1e32aa493d848cee54528e0f7b/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:25.427189 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.427525 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.427751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.463862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] MFA Rules not processed for user `157f4d1e32aa493d848cee54528e0f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:25.465296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:25.514248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:25.552846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1276/2553] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:25.557384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.601944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:25.643853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7118e9-92d5-4d34-94f9-52a6480da1f3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.648738 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.649358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.649639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.649913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.650555 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.652639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.686163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1278/2554] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:25 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:25.692440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.705484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b263d7e-358c-4065-8d37-1359323afccc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.768507 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.769127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.769409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.769683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.770319 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.772304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.797589 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1277/2555] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:25 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:25.803786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.818200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251848-6be5-4731-ba3a-457db0bbc4ba admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.823047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.823646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.823923 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.824698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.825353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.826977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.839557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1279/2556] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:25.844154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.858081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afebd1-659c-4baf-a19d-3b9ff7dfabe5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:25.866130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:25.866901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:25.867786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:25.867786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] PATH_INFO: `/v3/projects/a7d0a67ee0184e3a9489a9b5448e5901/users/1a923d6ad08647909347ad2bbe52e36e/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:25.909883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d0a67ee0184e3a9489a9b5448e5901, user_id=1a923d6ad08647909347ad2bbe52e36e, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:25.914010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:25.939055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1278/2557] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:25 2022] PUT /identity/v3/projects/a7d0a67ee0184e3a9489a9b5448e5901/users/1a923d6ad08647909347ad2bbe52e36e/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:25.940333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:25.960450 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc6350-3d4b-4323-9b9d-c790404e9fa2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:26.096821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b614d5-b379-4ccc-b77f-c4ad7c23d0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:26.098198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b614d5-b379-4ccc-b77f-c4ad7c23d0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:26.098779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b614d5-b379-4ccc-b77f-c4ad7c23d0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:26.099222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b614d5-b379-4ccc-b77f-c4ad7c23d0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:26.100073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b614d5-b379-4ccc-b77f-c4ad7c23d0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:26.103817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b614d5-b379-4ccc-b77f-c4ad7c23d0e0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:26.136735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1280/2558] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:26.137379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:26.151845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356313a2-19c0-4562-a203-21202838ff4c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:26.160907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:26.162284 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:26.163155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:26.163927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] PATH_INFO: `/v3/projects/a7d0a67ee0184e3a9489a9b5448e5901/users/1a923d6ad08647909347ad2bbe52e36e/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:26.168131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d0a67ee0184e3a9489a9b5448e5901, user_id=1a923d6ad08647909347ad2bbe52e36e, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:26.172400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:26.220674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1279/2559] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:26 2022] PUT /identity/v3/projects/a7d0a67ee0184e3a9489a9b5448e5901/users/1a923d6ad08647909347ad2bbe52e36e/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:26.224237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:26.224237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:26.224545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:26.283558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] MFA Rules not processed for user `1a923d6ad08647909347ad2bbe52e36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:26.287615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:26.397195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1281/2560] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:26.429637 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:26.432752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:26.564778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908664e-7bc2-4734-afae-434648651008 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:26.568834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308399cd-84d4-49f4-af4b-ce24aa98262a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:26.569542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308399cd-84d4-49f4-af4b-ce24aa98262a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:26.569891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308399cd-84d4-49f4-af4b-ce24aa98262a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:26.570240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308399cd-84d4-49f4-af4b-ce24aa98262a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:26.582967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308399cd-84d4-49f4-af4b-ce24aa98262a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:26.584648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308399cd-84d4-49f4-af4b-ce24aa98262a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:26.596817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1280/2561] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:26.624654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:26.627127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:26.668109 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:26.711074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917f18-ac08-4341-8fba-a214505ab3f3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:26.716093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:26.716870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:26.717228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:26.717643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] PATH_INFO: `/v3/users/25c6dde2c21e455682620ee1982a6581` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:26.718928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c6dde2c21e455682620ee1982a6581)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:26.722473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:26.819066 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c6dde2c21e455682620ee1982a6581'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:26.850361 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c6dde2c21e455682620ee1982a6581'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:26.874324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c6dde2c21e455682620ee1982a6581'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:26.875223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c6dde2c21e455682620ee1982a6581'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:26.876111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c6dde2c21e455682620ee1982a6581'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:26.901030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1282/2562] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:26 2022] DELETE /identity/v3/users/25c6dde2c21e455682620ee1982a6581 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:26.911278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:26.911278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:26.911278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:26.938379 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:26.940941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:26.978821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:27.070291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1281/2563] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:27.077571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:27.079852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.178962 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0765b-7a15-4caf-84e0-07393ee636a7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:27.182947 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7dc0a-db55-47a7-a6e6-10aea071993e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:27.183442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7dc0a-db55-47a7-a6e6-10aea071993e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:27.183680 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7dc0a-db55-47a7-a6e6-10aea071993e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:27.183956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7dc0a-db55-47a7-a6e6-10aea071993e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:27.197963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7dc0a-db55-47a7-a6e6-10aea071993e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:27.199439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7dc0a-db55-47a7-a6e6-10aea071993e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:27.214547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1283/2564] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:27.234552 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:27.237137 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.256006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206b90e-407a-411c-b983-55b0093987c1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:27.270429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:27.272821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:27.273378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:27.273896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] PATH_INFO: `/v3/users/157f4d1e32aa493d848cee54528e0f7b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:27.275778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=157f4d1e32aa493d848cee54528e0f7b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:27.280542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:27.368890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '157f4d1e32aa493d848cee54528e0f7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.369585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '157f4d1e32aa493d848cee54528e0f7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.371354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '157f4d1e32aa493d848cee54528e0f7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.393576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '157f4d1e32aa493d848cee54528e0f7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.403176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '157f4d1e32aa493d848cee54528e0f7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.418405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1282/2565] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:27 2022] DELETE /identity/v3/users/157f4d1e32aa493d848cee54528e0f7b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:27.425201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:27.425560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:27.426757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:27.463115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:27.464433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.566421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:27.600508 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1284/2566] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:27.613266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:27.621361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.653260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:27.655784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.684000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94facc0d-daa3-4499-829a-e4d77eaec9ff None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:27.689740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:27.691258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:27.691711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:27.692013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] PATH_INFO: `/v3/projects/23cbc7b2b92540a493a168e87b0e53bd` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:27.693438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23cbc7b2b92540a493a168e87b0e53bd)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:27.695799 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:27.770858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23cbc7b2b92540a493a168e87b0e53bd', 'request_id': 'req-edc25d0e-4e43-4217-a529-f5edfdcd3035'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.771377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174de364-b662-47bb-8d3e-ff160a916474 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:27.780028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23cbc7b2b92540a493a168e87b0e53bd', 'request_id': 'req-edc25d0e-4e43-4217-a529-f5edfdcd3035'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.793962 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23cbc7b2b92540a493a168e87b0e53bd', 'request_id': 'req-edc25d0e-4e43-4217-a529-f5edfdcd3035'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.806586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] The token cache is being invalidate because project 23cbc7b2b92540a493a168e87b0e53bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:27.806867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.807543 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:27.809220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1285/2567] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:27 2022] DELETE /identity/v3/projects/23cbc7b2b92540a493a168e87b0e53bd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:27.814008 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:27.816166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.836640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:27.837428 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:27.837636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:27.840746 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:27.842796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:27.859591 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:27.926421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc25d0e-4e43-4217-a529-f5edfdcd3035 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:27.933586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:27.934437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:27.935181 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:27.936238 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] PATH_INFO: `/v3/users/bf2ee6aa889941e2b1539ee45cd06849` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:27.936980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2ee6aa889941e2b1539ee45cd06849)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:27.940145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:27.949218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:27.950684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:27.965260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1283/2568] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:28.052313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2ee6aa889941e2b1539ee45cd06849'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.073774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2ee6aa889941e2b1539ee45cd06849'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.083898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2ee6aa889941e2b1539ee45cd06849'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.084587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2ee6aa889941e2b1539ee45cd06849'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.085261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2ee6aa889941e2b1539ee45cd06849'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.088271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:28.105924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1286/2569] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:27 2022] DELETE /identity/v3/users/bf2ee6aa889941e2b1539ee45cd06849 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:28.394768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:28.396795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:28.411227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcda390-47f3-48e6-b8be-e9a52c89fb6c service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:28.492470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:28.493111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:28.493386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:28.493658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] PATH_INFO: `/v3/projects/c4bf3a1988c4435680aec058007a438f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:28.494789 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4bf3a1988c4435680aec058007a438f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:28.496813 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:28.571481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4bf3a1988c4435680aec058007a438f', 'request_id': 'req-23d9578a-76a4-482f-bdb2-1b5de989e600'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.572239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:28.574310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:28.585998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4bf3a1988c4435680aec058007a438f', 'request_id': 'req-23d9578a-76a4-482f-bdb2-1b5de989e600'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.593909 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4bf3a1988c4435680aec058007a438f', 'request_id': 'req-23d9578a-76a4-482f-bdb2-1b5de989e600'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.602411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e25fb3-43cb-4ff0-924a-696b11714e23 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:28.609330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] The token cache is being invalidate because project c4bf3a1988c4435680aec058007a438f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:28.609629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.610482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.615226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1284/2570] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:28 2022] DELETE /identity/v3/projects/c4bf3a1988c4435680aec058007a438f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:28.616415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:28.618517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:28.668104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:28.669471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:28.670016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:28.670131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:28.670348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] PATH_INFO: `/v3/projects/a71642b5faa7468da0b6a31e6b225efb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:28.671760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71642b5faa7468da0b6a31e6b225efb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:28.674019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:28.692980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9578a-76a4-482f-bdb2-1b5de989e600 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:28.699975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:28.700797 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:28.701080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:28.701403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] PATH_INFO: `/v3/users/1a923d6ad08647909347ad2bbe52e36e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:28.702756 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a923d6ad08647909347ad2bbe52e36e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:28.705662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:28.735933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71642b5faa7468da0b6a31e6b225efb', 'request_id': 'req-4c807d10-669e-4d6a-8f0a-9b91b19252c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.743533 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71642b5faa7468da0b6a31e6b225efb', 'request_id': 'req-4c807d10-669e-4d6a-8f0a-9b91b19252c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.761835 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71642b5faa7468da0b6a31e6b225efb', 'request_id': 'req-4c807d10-669e-4d6a-8f0a-9b91b19252c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.776126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a923d6ad08647909347ad2bbe52e36e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.777084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a923d6ad08647909347ad2bbe52e36e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.777243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] The token cache is being invalidate because project a71642b5faa7468da0b6a31e6b225efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:28.777525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.777901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a923d6ad08647909347ad2bbe52e36e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.778117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c807d10-669e-4d6a-8f0a-9b91b19252c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.780237 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1287/2571] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:28 2022] DELETE /identity/v3/projects/a71642b5faa7468da0b6a31e6b225efb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:28.785951 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:28.786476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:28.787219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:28.796005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a923d6ad08647909347ad2bbe52e36e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.807240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c8a4c8c-d9c3-4603-94e1-b3dda6127fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a923d6ad08647909347ad2bbe52e36e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:28.811845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:28.814416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:28.834401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1285/2572] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:28 2022] DELETE /identity/v3/users/1a923d6ad08647909347ad2bbe52e36e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:28.887898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1288/2573] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:28.897011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:28.897579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:28.897963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:28.923086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:28.924854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:29.010825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1286/2574] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:29.019632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.030475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401808d2-9d6f-41ff-8c70-47c1a2d5f611 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.035889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.037126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.037126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.037126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.037749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:29.038051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.039669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.054600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1289/2575] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:29.059980 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.071808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b8515-1ae5-45f7-a2c0-b60625edb31a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.079471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.080263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.081010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.081010 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.081398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.083486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.120143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1287/2576] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:29.121943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.137768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baeef82-b4cd-4a85-a0f1-9d1412543e5b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.150135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:29.228716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.229490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.229821 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.230243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.231000 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.233438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.261179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1290/2577] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/users => generated 304 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:28:29.262415 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.274142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbac70-437e-4f28-9640-e227fc5e499f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.280145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.282644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.282775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.283037 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.283757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.285497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.299163 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1288/2578] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:29.304053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.315552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0649de-ad97-4926-815c-2ae19123ada7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.320652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe12d2-8b85-4d0c-8de0-b2580c4d1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.321272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe12d2-8b85-4d0c-8de0-b2580c4d1610 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.321537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe12d2-8b85-4d0c-8de0-b2580c4d1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.321811 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe12d2-8b85-4d0c-8de0-b2580c4d1610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.322542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe12d2-8b85-4d0c-8de0-b2580c4d1610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.324168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe12d2-8b85-4d0c-8de0-b2580c4d1610 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.338092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1291/2579] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:29.342341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.353540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f331c-a6b9-4231-be12-e51abbf9990b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.358990 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.359792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.360216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.360727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] PATH_INFO: `/v3/projects/8e4b7cdd38974e67a55982f4c1db1799/users/eca94ecfa23444aa8a41a3ff78a8cd8c/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.381054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4b7cdd38974e67a55982f4c1db1799, user_id=eca94ecfa23444aa8a41a3ff78a8cd8c, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.384342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.401963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1289/2580] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:29 2022] PUT /identity/v3/projects/8e4b7cdd38974e67a55982f4c1db1799/users/eca94ecfa23444aa8a41a3ff78a8cd8c/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:29.418163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.418546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.419456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.459055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] MFA Rules not processed for user `eca94ecfa23444aa8a41a3ff78a8cd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:29.460160 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.460359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:29.463355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:29.486258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:29.550253 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1292/2581] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:29.554644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.561767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae1514-eb63-42a2-b5a3-2c4cc0ac9b31 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.567378 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.568026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.568339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.568617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] PATH_INFO: `/v3/projects/a7d0a67ee0184e3a9489a9b5448e5901` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.569782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d0a67ee0184e3a9489a9b5448e5901)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.571707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.578814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a9792-41bf-418b-9392-a413653b9e0b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.587020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.587904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.588338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.588614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.589244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.591723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.655367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1293/2582] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:28:29.658251 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.660142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d0a67ee0184e3a9489a9b5448e5901', 'request_id': 'req-d0bd0a49-ae1b-4867-86ea-d74c3f516879'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:29.671541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d0a67ee0184e3a9489a9b5448e5901', 'request_id': 'req-d0bd0a49-ae1b-4867-86ea-d74c3f516879'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:29.681893 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d0a67ee0184e3a9489a9b5448e5901', 'request_id': 'req-d0bd0a49-ae1b-4867-86ea-d74c3f516879'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:29.695923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] The token cache is being invalidate because project a7d0a67ee0184e3a9489a9b5448e5901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:29.696471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:29.698110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0bd0a49-ae1b-4867-86ea-d74c3f516879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:29.702386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1290/2583] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:29 2022] DELETE /identity/v3/projects/a7d0a67ee0184e3a9489a9b5448e5901 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:29.708616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484595-fb41-4c73-abdb-13fbad29919b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.708906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484595-fb41-4c73-abdb-13fbad29919b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.709187 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484595-fb41-4c73-abdb-13fbad29919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.739725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6c484595-fb41-4c73-abdb-13fbad29919b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:29.739725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6c484595-fb41-4c73-abdb-13fbad29919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:29.744958 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:29.750160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7fa17-421b-45c3-aa55-7adf82867880 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.756941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.757564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.757564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.757564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.757564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.760846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.784885 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1294/2584] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/users => generated 309 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:29.788945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.790732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:29.806740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10baf7-3861-4cae-8324-a87a33cf85ee admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.814186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.814930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.815233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.815504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.816152 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.817533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1291/2585] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:29.817836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.828127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.829446 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.829631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1295/2586] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:29.829780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.834317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.846005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a5050-dffe-47e3-b026-222122eb8cde admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.850600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.851119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:29.851265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.851700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.851996 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] PATH_INFO: `/v3/projects/2162b3b32a664069b321ed8431d0a51a/users/4ce49b1e780d4b3d8995dd3253a2c646/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.852408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:29.861718 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1292/2587] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:28:29.866871 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.875898 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2162b3b32a664069b321ed8431d0a51a, user_id=4ce49b1e780d4b3d8995dd3253a2c646, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.878197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6146db-1bae-4b45-8c52-fb7c7f5a4c8a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.879302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.883586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.884094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.884437 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.884734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.885389 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:29.885646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.887335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:29.904733 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1296/2588] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:29 2022] PUT /identity/v3/projects/2162b3b32a664069b321ed8431d0a51a/users/4ce49b1e780d4b3d8995dd3253a2c646/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:29.905206 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1293/2589] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:28:29.911380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.911637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:29.923818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418729-3a52-492a-84ae-f641b1924575 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.940235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:29.980623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae78f-efd9-4ab6-9b0b-1cd5f8682fc1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:29.986641 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:29.987260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:29.987539 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:29.987809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:29.988480 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:29.990407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.006441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.008454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.008816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.009145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.009844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.011495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.023064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1294/2590] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:30.027253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.031653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1297/2591] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:29 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:30.033438 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.039879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f768c3e-7d43-434b-adc6-d71dbd3f96cd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.109760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82b7b-797a-4186-8c37-676c110bd196 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.115022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.115232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb499a81-f2f3-4ebf-b54a-2f1c027d7b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.115673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb499a81-f2f3-4ebf-b54a-2f1c027d7b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.115806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.115953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb499a81-f2f3-4ebf-b54a-2f1c027d7b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.116052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.116255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb499a81-f2f3-4ebf-b54a-2f1c027d7b6f admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.116404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] PATH_INFO: `/v3/projects/2162b3b32a664069b321ed8431d0a51a/users/4ce49b1e780d4b3d8995dd3253a2c646/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.116888 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb499a81-f2f3-4ebf-b54a-2f1c027d7b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.118393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2162b3b32a664069b321ed8431d0a51a, user_id=4ce49b1e780d4b3d8995dd3253a2c646, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.118854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb499a81-f2f3-4ebf-b54a-2f1c027d7b6f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.121177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.135027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1295/2592] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:30 2022] PUT /identity/v3/projects/2162b3b32a664069b321ed8431d0a51a/users/4ce49b1e780d4b3d8995dd3253a2c646/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:30.144580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1298/2593] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:30 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:30.148626 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.150374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.150746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.151093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.183097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] MFA Rules not processed for user `4ce49b1e780d4b3d8995dd3253a2c646`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:30.184511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:30.199114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:30.227731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef02540-2d93-4aea-9f67-ca0d665937b3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.238313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.238933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.239213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.239484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.240132 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.242604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.256029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1296/2594] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:30.260543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.273866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1299/2595] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:30.276654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ffd55-9ce0-44c8-8942-a4f102d91614 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.282655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.283268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.283725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.283993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.284811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.287193 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.288424 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.290719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:30.300764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1297/2596] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:30.305315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.322309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1daed9-b8f5-4188-b412-9af8ddfded81 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.331573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcff8f8-9529-4303-a129-d6a8ced1fddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.332643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff8f8-9529-4303-a129-d6a8ced1fddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.333093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff8f8-9529-4303-a129-d6a8ced1fddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.333951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff8f8-9529-4303-a129-d6a8ced1fddf admin admin] PATH_INFO: `/v3/projects/7ef17a4f746c4edd97e85920dbee5bc6/users/2279530d22c948839be7357dfc2554fb/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.359787 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcff8f8-9529-4303-a129-d6a8ced1fddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef17a4f746c4edd97e85920dbee5bc6, user_id=2279530d22c948839be7357dfc2554fb, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.365351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcff8f8-9529-4303-a129-d6a8ced1fddf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.388359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc47a6-0d8f-4b37-9b1a-b78bdbd9f4e1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.388829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1298/2597] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:30 2022] PUT /identity/v3/projects/7ef17a4f746c4edd97e85920dbee5bc6/users/2279530d22c948839be7357dfc2554fb/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:30.400435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.400821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.401180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.418275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:30.436817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] MFA Rules not processed for user `2279530d22c948839be7357dfc2554fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:30.438276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:30.457623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.458301 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.458576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.458845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.518369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1299/2598] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:30.519571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.552313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.553904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.565105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1300/2599] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:30 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:30.599872 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaa526-f7ec-4755-9a30-db4c85e35ad4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.604691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.605307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.605587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.605857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.606499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.608477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.649086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1300/2600] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:30 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:30.654635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.665812 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f430d6-7a8b-4575-93df-b7982e70aaf7 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.679053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:30.731515 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.731681 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.731963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.732265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.732890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.734825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.738128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.764245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1301/2601] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:30 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:30.768705 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.788822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015b7cd-eb7c-42a5-ac49-4303fe01c57c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.804699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.805589 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.806447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.807094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.808098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.810248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.818001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a2f3e-ef60-4b6d-aaba-d25b5b8fb72f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.823471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.823656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1302/2602] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:30.824631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.825197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.825784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.827664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.829171 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:30.839026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771870c6-ca9e-49c3-a775-5ec4340ef3f4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:30.844971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:30.845683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:30.846003 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:30.846375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] PATH_INFO: `/v3/projects/88c49f08d2f7403882c4ce88ab12fb9c/users/2c3b41630a874c91aeaa598a648ab563/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:30.869475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:30.873116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c49f08d2f7403882c4ce88ab12fb9c, user_id=2c3b41630a874c91aeaa598a648ab563, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.876568 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.898567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1303/2603] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:30 2022] PUT /identity/v3/projects/88c49f08d2f7403882c4ce88ab12fb9c/users/2c3b41630a874c91aeaa598a648ab563/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:30.903488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:30.915566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:30.917239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:30.928287 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1301/2604] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:30 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:28:31.019225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcb23e-1cb7-43f1-b886-9ca18fec8191 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:31.026097 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:31.027381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:31.027682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:31.028600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:31.029266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:31.031038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:31.047333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1304/2605] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:31.049267 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:31.062066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362976-b59b-40da-bcc1-3c57d19d2a16 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:31.068149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40135e-3067-4e3d-aed0-11e596020b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:31.068855 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40135e-3067-4e3d-aed0-11e596020b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:31.069136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40135e-3067-4e3d-aed0-11e596020b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:31.069409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40135e-3067-4e3d-aed0-11e596020b40 admin admin] PATH_INFO: `/v3/projects/88c49f08d2f7403882c4ce88ab12fb9c/users/2c3b41630a874c91aeaa598a648ab563/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:31.071554 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40135e-3067-4e3d-aed0-11e596020b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c49f08d2f7403882c4ce88ab12fb9c, user_id=2c3b41630a874c91aeaa598a648ab563, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:31.074265 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40135e-3067-4e3d-aed0-11e596020b40 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:31.092892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:31.095331 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:31.097498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1302/2606] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:31 2022] PUT /identity/v3/projects/88c49f08d2f7403882c4ce88ab12fb9c/users/2c3b41630a874c91aeaa598a648ab563/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:31.109090 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:31.109460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:31.109798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:31.144331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] MFA Rules not processed for user `2c3b41630a874c91aeaa598a648ab563`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:31.145502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:31.180497 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e69b-a705-487d-9a69-4baae4185b06 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:31.207274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:31.234720 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1303/2607] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:28:31.246100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:31.246726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:31.247005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:31.247273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:31.249295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:31.249662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:31.250863 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:31.337011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15358232-ee6a-4e4a-8e9b-ce662b16f289 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:31.342732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:31.343349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:31.343630 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:31.343904 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:31.354844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:31.356467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:31.367920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1304/2608] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:28:31.378642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:31.382020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:31.398087 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1305/2609] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:28:31.484573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:31.499972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd86c-2ca4-4254-9f53-1f69eeae90df service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:31.507226 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:31.508424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:31.509018 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:31.509574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:31.583641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:31.598065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:31.599679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:31.619727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1305/2610] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:38.684667 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:38.687235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:38.709601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543f6c-bb81-4ef0-bc0d-6960dbc7de48 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:38.718761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:38.719582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:38.719904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:38.720257 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] PATH_INFO: `/v3/users/7ea15f54d31149ea948650cdd363d42f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:38.722132 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea15f54d31149ea948650cdd363d42f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:38.725636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:38.785463 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea15f54d31149ea948650cdd363d42f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:38.798573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea15f54d31149ea948650cdd363d42f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:38.807946 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea15f54d31149ea948650cdd363d42f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:38.808809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea15f54d31149ea948650cdd363d42f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:38.809562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea15f54d31149ea948650cdd363d42f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:38.823551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1306/2611] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:38 2022] DELETE /identity/v3/users/7ea15f54d31149ea948650cdd363d42f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:39.049276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:39.065423 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2d8a8-516b-4c09-940a-db03f199fbf4 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:39.129674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:39.130353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:39.130633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:39.130999 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] PATH_INFO: `/v3/projects/ede6199069114ce380f30088d312f345` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:39.136227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede6199069114ce380f30088d312f345)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:39.136227 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:39.183848 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede6199069114ce380f30088d312f345', 'request_id': 'req-cde35043-fe24-400e-8b3d-ea6a39d23217'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.192110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede6199069114ce380f30088d312f345', 'request_id': 'req-cde35043-fe24-400e-8b3d-ea6a39d23217'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.198919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede6199069114ce380f30088d312f345', 'request_id': 'req-cde35043-fe24-400e-8b3d-ea6a39d23217'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.210568 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] The token cache is being invalidate because project ede6199069114ce380f30088d312f345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:39.210852 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.211504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.213016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1306/2612] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:39 2022] DELETE /identity/v3/projects/ede6199069114ce380f30088d312f345 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:39.217100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:39.218546 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:39.232367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab3f26-7742-4ce0-9722-d10d0ece6b66 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:39.277242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:39.294906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:39.295573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:39.295857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:39.296130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] PATH_INFO: `/v3/users/3630f86adf69408396086978eda5ee58` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:39.314466 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3630f86adf69408396086978eda5ee58)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:39.316460 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:39.372129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3630f86adf69408396086978eda5ee58'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.385785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3630f86adf69408396086978eda5ee58'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.394810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3630f86adf69408396086978eda5ee58'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.395492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3630f86adf69408396086978eda5ee58'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.396113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3630f86adf69408396086978eda5ee58'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.409702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1307/2613] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:39 2022] DELETE /identity/v3/users/3630f86adf69408396086978eda5ee58 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:39.598791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:39.600273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:39.618666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde35043-fe24-400e-8b3d-ea6a39d23217 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:39.702177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:39.702811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:39.703086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:39.703353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] PATH_INFO: `/v3/projects/f4fdcdd025f84466b5c5c9fd4ac5234d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:39.716067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fdcdd025f84466b5c5c9fd4ac5234d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:39.718092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:39.731709 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:39.769614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fdcdd025f84466b5c5c9fd4ac5234d', 'request_id': 'req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.781399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fdcdd025f84466b5c5c9fd4ac5234d', 'request_id': 'req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.785806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fdcdd025f84466b5c5c9fd4ac5234d', 'request_id': 'req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.796323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] The token cache is being invalidate because project f4fdcdd025f84466b5c5c9fd4ac5234d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:39.796612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.797411 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6cd4b0d-d426-42e1-8c94-8c1233b6b2b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:39.798919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1307/2614] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:39 2022] DELETE /identity/v3/projects/f4fdcdd025f84466b5c5c9fd4ac5234d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:45.082862 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:45.084437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:45.104135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e8ca-8cf4-4171-855c-c5c3d30219fc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:45.179229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:45.179913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:45.180200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:45.180489 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] PATH_INFO: `/v3/users/eca94ecfa23444aa8a41a3ff78a8cd8c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:45.181661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca94ecfa23444aa8a41a3ff78a8cd8c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:45.183605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:45.234353 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca94ecfa23444aa8a41a3ff78a8cd8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.240553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:45.249481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca94ecfa23444aa8a41a3ff78a8cd8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.257510 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca94ecfa23444aa8a41a3ff78a8cd8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.258125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca94ecfa23444aa8a41a3ff78a8cd8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.258761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca94ecfa23444aa8a41a3ff78a8cd8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.271564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1308/2615] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:45 2022] DELETE /identity/v3/users/eca94ecfa23444aa8a41a3ff78a8cd8c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:45.276257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:45.276947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:45.276947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:45.298607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:45.299851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:45.370145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1308/2616] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:45.380248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:45.381687 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:45.460487 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cc75c-d411-4c03-bd62-cac97c7ce716 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:45.464376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:45.464995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:45.465272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:45.465542 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:45.476654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:45.478101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:45.497033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1309/2617] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:45.804303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:45.806576 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:45.830956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669793e-e1ef-4d56-8ac0-cf3a760dd339 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:45.839111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:45.840145 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:45.841579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:45.841918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] PATH_INFO: `/v3/projects/8e4b7cdd38974e67a55982f4c1db1799` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:45.843436 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4b7cdd38974e67a55982f4c1db1799)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:45.845501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:45.903072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4b7cdd38974e67a55982f4c1db1799', 'request_id': 'req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.917312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4b7cdd38974e67a55982f4c1db1799', 'request_id': 'req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.931515 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4b7cdd38974e67a55982f4c1db1799', 'request_id': 'req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.952970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] The token cache is being invalidate because project 8e4b7cdd38974e67a55982f4c1db1799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:45.953998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.955146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:45.957533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1309/2618] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:45 2022] DELETE /identity/v3/projects/8e4b7cdd38974e67a55982f4c1db1799 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:45.962226 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:45.978170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e9c0c-9467-4f95-a3f4-faaedabbffc7 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:46.067022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:46.067672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:46.067938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:46.068222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] PATH_INFO: `/v3/users/4ce49b1e780d4b3d8995dd3253a2c646` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:46.069431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce49b1e780d4b3d8995dd3253a2c646)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:46.071434 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:46.134891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce49b1e780d4b3d8995dd3253a2c646'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.147724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce49b1e780d4b3d8995dd3253a2c646'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.155467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce49b1e780d4b3d8995dd3253a2c646'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.156096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce49b1e780d4b3d8995dd3253a2c646'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.157026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3f71b074-647d-49c1-b186-f5a5d83b762a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce49b1e780d4b3d8995dd3253a2c646'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.161788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:46.171842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1310/2619] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:45 2022] DELETE /identity/v3/users/4ce49b1e780d4b3d8995dd3253a2c646 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:46.483617 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:46.486528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:46.517903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e81246-324e-4b0b-9f9a-4e4ec595fb92 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:46.543966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:46.587196 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:46.587847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:46.588123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:46.588433 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] PATH_INFO: `/v3/projects/2162b3b32a664069b321ed8431d0a51a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:46.589585 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2162b3b32a664069b321ed8431d0a51a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:46.591541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:46.648818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2162b3b32a664069b321ed8431d0a51a', 'request_id': 'req-97586a45-6ba3-402b-94d1-89c0cd29dc91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.657518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2162b3b32a664069b321ed8431d0a51a', 'request_id': 'req-97586a45-6ba3-402b-94d1-89c0cd29dc91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.665107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2162b3b32a664069b321ed8431d0a51a', 'request_id': 'req-97586a45-6ba3-402b-94d1-89c0cd29dc91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.676569 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] The token cache is being invalidate because project 2162b3b32a664069b321ed8431d0a51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:46.676857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.677529 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97586a45-6ba3-402b-94d1-89c0cd29dc91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:46.679021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1310/2620] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:46 2022] DELETE /identity/v3/projects/2162b3b32a664069b321ed8431d0a51a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:46.684743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:46.685030 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:46.685303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:46.705080 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:46.706310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:46.778443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1311/2621] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:46.787493 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:46.788496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:46.788496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:46.813553 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:46.814778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:46.825519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1311/2622] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:46.831147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:46.845954 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce33bd7-2d6c-4b9e-81a1-717a143ee0c6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:46.850724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:46.851459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:46.851603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:46.851881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:46.852562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:46.852861 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:46.854505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:46.867837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1312/2623] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 11:28:46.872281 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:46.882527 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4eb56b-37b0-4525-a03e-4cee77e3d595 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:46.887505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:46.888214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:46.888563 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:46.888925 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:46.889721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:46.892240 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:46.924851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1312/2624] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:46 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:46.930682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:46.941760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ed486-6e07-457a-ad5e-7938f7ae5c5b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:46.944514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:46.946286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.019955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.020629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.020907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.021183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.021818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.023761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.025551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade09eb-9736-411a-a471-a3668ca59ca1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.033944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.035480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.035773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.036059 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] PATH_INFO: `/v3/users/2279530d22c948839be7357dfc2554fb` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.038308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2279530d22c948839be7357dfc2554fb)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.040561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.054855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1313/2625] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:46 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:47.058143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.072495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635db63-8ebf-42d2-9e53-15229adc4115 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.075890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:47.077962 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.078900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.079204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.079549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.080478 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.082420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.093915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1314/2626] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:47.098309 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.105941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2279530d22c948839be7357dfc2554fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.106636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2279530d22c948839be7357dfc2554fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.107288 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2279530d22c948839be7357dfc2554fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.108971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad918b6e-26b3-4c2c-b48f-5e3527fa085a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.115941 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.116272 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.116563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.116841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.117493 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.119099 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.121464 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2279530d22c948839be7357dfc2554fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.131286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1315/2627] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:47.131862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-951eecb8-6b6b-462e-b00e-af6903ff4de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2279530d22c948839be7357dfc2554fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.137183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.141121 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.151427 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1313/2628] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:46 2022] DELETE /identity/v3/users/2279530d22c948839be7357dfc2554fb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:47.155938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.156876 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.157368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782566f-d042-4845-92ac-558c0813b359 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.157572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.186694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:47.188502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.213185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:47.229912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986b961-26d2-46c8-b7c1-f90e538c7298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.230570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986b961-26d2-46c8-b7c1-f90e538c7298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.231709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986b961-26d2-46c8-b7c1-f90e538c7298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.231986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986b961-26d2-46c8-b7c1-f90e538c7298 admin admin] PATH_INFO: `/v3/projects/34db09cf16f14a49b4d87b4be5aafdb6/users/ddfebf61632d4efdb04269f7c9e2f654/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.255752 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986b961-26d2-46c8-b7c1-f90e538c7298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34db09cf16f14a49b4d87b4be5aafdb6, user_id=ddfebf61632d4efdb04269f7c9e2f654, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.258462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986b961-26d2-46c8-b7c1-f90e538c7298 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.263088 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1314/2629] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:28:47.274825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.276503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.283693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1316/2630] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:47 2022] PUT /identity/v3/projects/34db09cf16f14a49b4d87b4be5aafdb6/users/ddfebf61632d4efdb04269f7c9e2f654/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:47.298459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.298916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.299269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.341269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] MFA Rules not processed for user `ddfebf61632d4efdb04269f7c9e2f654`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:47.342545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.361656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e59855-ef67-4f0a-b756-8998973ecf30 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.417404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1317/2631] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:28:47.418360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.419185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.419482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.419772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.429999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.431469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.434548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:47.503690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.505099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.515920 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1315/2632] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:47.553627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7269352-13d1-4564-a97a-38f3d1773193 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.558125 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e78415ea98940bbbc59c733c8ffefa4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.558866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.559222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.559572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.569326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.570778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.581453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1318/2633] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:28:47.851084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.852779 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:47.868253 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9000d-e8dd-4620-97c9-98d28d4ca9da service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:47.874245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:47.874982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:47.875399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:47.875741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] PATH_INFO: `/v3/projects/7ef17a4f746c4edd97e85920dbee5bc6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:47.877073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef17a4f746c4edd97e85920dbee5bc6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:47.879093 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:47.940752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef17a4f746c4edd97e85920dbee5bc6', 'request_id': 'req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.948984 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef17a4f746c4edd97e85920dbee5bc6', 'request_id': 'req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.958184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef17a4f746c4edd97e85920dbee5bc6', 'request_id': 'req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.972530 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] The token cache is being invalidate because project 7ef17a4f746c4edd97e85920dbee5bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:47.972878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.974304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:47.976905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1316/2634] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:47 2022] DELETE /identity/v3/projects/7ef17a4f746c4edd97e85920dbee5bc6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:47.981553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:47.995598 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d884e5-81b9-4b57-89c2-0bd837b6fa59 service glance] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:48.049159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:48.067712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:48.068382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:48.068656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:48.069614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] PATH_INFO: `/v3/users/2c3b41630a874c91aeaa598a648ab563` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:48.070100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c3b41630a874c91aeaa598a648ab563)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:48.072104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:48.147791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c3b41630a874c91aeaa598a648ab563'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.163686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c3b41630a874c91aeaa598a648ab563'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.173176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c3b41630a874c91aeaa598a648ab563'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.174062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c3b41630a874c91aeaa598a648ab563'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.175239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c829fdb-bc1a-44f8-b2d1-608519d7333f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c3b41630a874c91aeaa598a648ab563'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.189395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1319/2635] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:47 2022] DELETE /identity/v3/users/2c3b41630a874c91aeaa598a648ab563 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:48.533804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:48.537204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:48.566144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb63f-46b8-4efc-94e5-a6f5dbfe152a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:48.635989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:48.636666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:48.636950 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:48.637346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] PATH_INFO: `/v3/projects/88c49f08d2f7403882c4ce88ab12fb9c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:48.638668 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c49f08d2f7403882c4ce88ab12fb9c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:48.641254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:48.684697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:48.695700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c49f08d2f7403882c4ce88ab12fb9c', 'request_id': 'req-9bdc512d-464a-440e-ad19-fb15cecabc03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.707233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c49f08d2f7403882c4ce88ab12fb9c', 'request_id': 'req-9bdc512d-464a-440e-ad19-fb15cecabc03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.719926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c49f08d2f7403882c4ce88ab12fb9c', 'request_id': 'req-9bdc512d-464a-440e-ad19-fb15cecabc03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.730845 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] The token cache is being invalidate because project 88c49f08d2f7403882c4ce88ab12fb9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:48.731119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.731794 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bdc512d-464a-440e-ad19-fb15cecabc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:48.733306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1317/2636] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:48 2022] DELETE /identity/v3/projects/88c49f08d2f7403882c4ce88ab12fb9c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:48.739035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:48.739363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:48.739601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:48.760176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:48.761453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:48.832308 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1320/2637] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:48.840302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:48.840588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:48.840862 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:48.865098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:48.866613 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:48.877286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1318/2638] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:48.881614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:48.892606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3c81e-4c37-48b4-94d5-a579a937cf0d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:48.897388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:48.897999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:48.898273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:48.898545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:48.899193 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:48.899492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:48.901157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:48.917872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1321/2639] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:48.923938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:48.934180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf440efc-3dc4-49c9-b8f8-c742bfec786d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:48.939351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:48.939952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:48.940258 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:48.940528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:48.941247 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:48.943261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:48.981169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1319/2640] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:48 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:48.986628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.001175 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b20fe5-a80e-4b7e-9c45-92110a7ce243 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.062123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:49.074395 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.075004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.075283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.075552 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.076183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.078177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.105244 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1322/2641] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:48 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:49.107045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.119232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609aa2a-9210-445c-8cd3-b0961ef5b24f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.123903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.124570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.124811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.125741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.126595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.128506 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.139158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1320/2642] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:49.143284 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.153618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c394d-d3ae-49d4-9478-5f3c88cd8c76 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.160746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd0da1-4568-4ae5-b36c-86d5fe41085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.161706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd0da1-4568-4ae5-b36c-86d5fe41085d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.162159 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd0da1-4568-4ae5-b36c-86d5fe41085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.162601 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd0da1-4568-4ae5-b36c-86d5fe41085d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.163706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd0da1-4568-4ae5-b36c-86d5fe41085d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.166540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd0da1-4568-4ae5-b36c-86d5fe41085d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.187782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1323/2643] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:49.192696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.206192 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836fc0-6887-4b5d-a06d-c96b5e1bd3d9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.211125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.211796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.212074 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.212377 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] PATH_INFO: `/v3/projects/fef5fdf8ed0b4c87b6e76fdf68b8754f/users/90ec146708a849f7b0a0d12f2de9abca/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.236234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef5fdf8ed0b4c87b6e76fdf68b8754f, user_id=90ec146708a849f7b0a0d12f2de9abca, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.236234 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.250967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1321/2644] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:49 2022] PUT /identity/v3/projects/fef5fdf8ed0b4c87b6e76fdf68b8754f/users/90ec146708a849f7b0a0d12f2de9abca/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:49.261511 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.261796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.262070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.292370 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] MFA Rules not processed for user `90ec146708a849f7b0a0d12f2de9abca`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:49.293642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:49.365491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1324/2645] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:49.370084 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.447208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62279d9b-3b89-4014-91d8-e59aeb94e840 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.452354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.452961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.453884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.454341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.454839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.456732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.491571 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1322/2646] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:49 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:49.492868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.504666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed94a3-2817-464d-9567-a57f53d9d488 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.586403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.587204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.587591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.587957 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.588853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.591362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.623915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1325/2647] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:49 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:28:49.625798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.636704 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedfb57-bdf8-4341-b281-5cd7bf16cb59 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.642320 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.642947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.643225 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.643522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.644169 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.645828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.658812 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1323/2648] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:49.662908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.678999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261630d-4409-41ce-9854-4c9ed69bc302 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.683611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.688317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.688317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.688317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] PATH_INFO: `/v3/projects/a5370cf0f65344519764619113d11f0b/users/fe9e521012b6435e948696bb6cef6a35/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.703426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5370cf0f65344519764619113d11f0b, user_id=fe9e521012b6435e948696bb6cef6a35, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.706550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.721557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1326/2649] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:49 2022] PUT /identity/v3/projects/a5370cf0f65344519764619113d11f0b/users/fe9e521012b6435e948696bb6cef6a35/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:49.728112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.739923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1be6e6-ede6-48b6-8011-486903df0aa6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.785021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:49.821951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41233c-8f26-473d-bde0-71b9ea778cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.822573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41233c-8f26-473d-bde0-71b9ea778cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.822856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41233c-8f26-473d-bde0-71b9ea778cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.823131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41233c-8f26-473d-bde0-71b9ea778cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.823784 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41233c-8f26-473d-bde0-71b9ea778cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.825456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41233c-8f26-473d-bde0-71b9ea778cd0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.836921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1324/2650] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:28:49.841275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:49.852717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61e1e3-9c01-448c-9e29-12a4454149e9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:49.857526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:49.858266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.858612 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.858913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] PATH_INFO: `/v3/projects/a5370cf0f65344519764619113d11f0b/users/fe9e521012b6435e948696bb6cef6a35/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.861482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5370cf0f65344519764619113d11f0b, user_id=fe9e521012b6435e948696bb6cef6a35, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:49.864292 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:49.876998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1327/2651] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:49 2022] PUT /identity/v3/projects/a5370cf0f65344519764619113d11f0b/users/fe9e521012b6435e948696bb6cef6a35/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:49.887690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:49.887978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:49.888294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:49.920854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] MFA Rules not processed for user `fe9e521012b6435e948696bb6cef6a35`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:49.922216 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:50.003819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1325/2652] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:50.016841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:50.018662 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:50.039842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:50.090779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760838e-c334-4296-9935-0fafa5ab92f9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:50.095304 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:50.095933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:50.096233 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:50.096520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:50.106404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:50.107837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:50.118062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1328/2653] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:50.503466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:50.513610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a162a-bf04-4e6a-9533-f980767eab4f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:50.518947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9e521012b6435e948696bb6cef6a35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5370cf0f65344519764619113d11f0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:50.519567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:50.519853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:50.520131 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:50.520835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:50.521140 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:50.522775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:50.536176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1326/2654] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:50.540777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:50.551105 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d40e4-0326-4c82-a637-4ba0935178c8 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:50.556551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9e521012b6435e948696bb6cef6a35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5370cf0f65344519764619113d11f0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:50.557252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:50.557540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:50.557817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:50.558492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:50.560522 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:50.597675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1329/2655] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:50 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:50.602961 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:50.618370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242ae07-7d84-4016-b75a-6de5cd78664b tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:50.683939 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9e521012b6435e948696bb6cef6a35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5370cf0f65344519764619113d11f0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:50.684705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:50.684890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:50.685168 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:50.685825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:50.686134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:50.687757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:50.700491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1327/2656] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:50.740299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:50.750771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3937-c912-4169-83d2-104bc101c2f1 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:50.755570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe9e521012b6435e948696bb6cef6a35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5370cf0f65344519764619113d11f0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:50.756232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:50.756519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:50.756792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] PATH_INFO: `/v3/projects/93a4758e0aa5486e9e46bc6e48ca3daa` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:50.757931 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorizing `identity:delete_project(project_id=93a4758e0aa5486e9e46bc6e48ca3daa)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:50.759903 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:50.827095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a4758e0aa5486e9e46bc6e48ca3daa', 'request_id': 'req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:50.839015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a4758e0aa5486e9e46bc6e48ca3daa', 'request_id': 'req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:50.853123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a4758e0aa5486e9e46bc6e48ca3daa', 'request_id': 'req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:50.867656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] The token cache is being invalidate because project 93a4758e0aa5486e9e46bc6e48ca3daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:50.867656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:50.867974 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6e40a326-9c8d-4ab4-8279-d78c7cf33a8a tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:50.869547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1330/2657] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:50 2022] DELETE /identity/v3/projects/93a4758e0aa5486e9e46bc6e48ca3daa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:51.112728 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:51.114523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:51.181847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:51.216602 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6c86-3bbd-4ef5-90ca-af06708edf62 tempest-VolumeQuotaClassesTest-853657287 tempest-VolumeQuotaClassesTest-853657287-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:51.293694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:51.294342 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:51.294930 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:51.295147 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] PATH_INFO: `/v3/users/90ec146708a849f7b0a0d12f2de9abca` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:51.296207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ec146708a849f7b0a0d12f2de9abca)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:51.299213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:51.385903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ec146708a849f7b0a0d12f2de9abca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:51.386586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ec146708a849f7b0a0d12f2de9abca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:51.387209 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ec146708a849f7b0a0d12f2de9abca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:51.404329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ec146708a849f7b0a0d12f2de9abca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:51.417051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ec146708a849f7b0a0d12f2de9abca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:51.441910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1328/2658] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:51 2022] DELETE /identity/v3/users/90ec146708a849f7b0a0d12f2de9abca => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:51.444728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:51.445134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:51.445502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:51.470995 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:51.472642 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:51.544496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1331/2659] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:51.555874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:51.558001 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:51.635484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f540b-9a5c-4767-a42d-2182104874d9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:51.649349 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:51.650138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:51.650572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:51.651007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:51.664624 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:51.665912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:51.677780 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1329/2660] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:28:51.986538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:51.987988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:52.006956 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3add9f-03f4-4180-9f6e-4ef31424a335 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:52.021217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:52.021886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:52.022168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:52.022445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] PATH_INFO: `/v3/projects/fef5fdf8ed0b4c87b6e76fdf68b8754f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:52.023746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef5fdf8ed0b4c87b6e76fdf68b8754f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:52.027633 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:52.039582 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:52.082401 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef5fdf8ed0b4c87b6e76fdf68b8754f', 'request_id': 'req-f09d9cc4-b606-4e87-b769-19fb2cc25459'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.087967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef5fdf8ed0b4c87b6e76fdf68b8754f', 'request_id': 'req-f09d9cc4-b606-4e87-b769-19fb2cc25459'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.097167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef5fdf8ed0b4c87b6e76fdf68b8754f', 'request_id': 'req-f09d9cc4-b606-4e87-b769-19fb2cc25459'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.110571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] The token cache is being invalidate because project fef5fdf8ed0b4c87b6e76fdf68b8754f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:52.110880 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.111560 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.113117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1332/2661] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:51 2022] DELETE /identity/v3/projects/fef5fdf8ed0b4c87b6e76fdf68b8754f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:52.117264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:52.127622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab47c-8923-49c8-9baf-10f6be4f7018 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:52.131294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:52.195707 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:52.196369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:52.196648 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:52.196921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] PATH_INFO: `/v3/users/fe9e521012b6435e948696bb6cef6a35` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:52.198042 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9e521012b6435e948696bb6cef6a35)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:52.200183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:52.258468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9e521012b6435e948696bb6cef6a35'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.259136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9e521012b6435e948696bb6cef6a35'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.259773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9e521012b6435e948696bb6cef6a35'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.280982 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9e521012b6435e948696bb6cef6a35'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.290222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f472b30c-5cd6-4fce-99d5-9c6ad2039805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9e521012b6435e948696bb6cef6a35'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:52.307676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1330/2662] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:28:52 2022] DELETE /identity/v3/users/fe9e521012b6435e948696bb6cef6a35 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:28:52.944844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:52.946669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:52.962365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d9cc4-b606-4e87-b769-19fb2cc25459 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.033881 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.034528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.034810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.035085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] PATH_INFO: `/v3/projects/a5370cf0f65344519764619113d11f0b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.036286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5370cf0f65344519764619113d11f0b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.038267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.097322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5370cf0f65344519764619113d11f0b', 'request_id': 'req-b1a99204-02a8-4e85-84c9-f366bde8e9ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:53.103203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5370cf0f65344519764619113d11f0b', 'request_id': 'req-b1a99204-02a8-4e85-84c9-f366bde8e9ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:53.111385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5370cf0f65344519764619113d11f0b', 'request_id': 'req-b1a99204-02a8-4e85-84c9-f366bde8e9ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:53.123765 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] The token cache is being invalidate because project a5370cf0f65344519764619113d11f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:28:53.124068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:53.124750 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b1a99204-02a8-4e85-84c9-f366bde8e9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:28:53.127100 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1333/2663] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:28:52 2022] DELETE /identity/v3/projects/a5370cf0f65344519764619113d11f0b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:53.132634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.133162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.133162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.155711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:53.157361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:53.248573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1331/2664] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:53.257786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.258072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.258354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.278248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:53.279485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:53.288615 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1334/2665] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:28:53.292997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.304236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689bee7-9fc7-4564-95b4-c87ed22c69f3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.309307 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.309907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.310184 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.310494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.311096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:28:53.311398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.313387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.325662 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1332/2666] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:28:53.331108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.347176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11469945-bbb5-4319-9576-a1373d8166fb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.352698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.353769 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.354286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.354790 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.355932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.357930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.389558 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:53.398986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1335/2667] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:28:53 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:53.402744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.414130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe87a2-7f48-4905-946f-af02bc4773a4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.423505 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:28:53.496117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.496772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.497056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.497422 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.498065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.500044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.523364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1333/2668] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:28:53 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:28:53.527554 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.548058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd52ad5-a6a1-479a-a2ae-a622a413dfe0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.549907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.551024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.551495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.551929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.553145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.556347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.571332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1336/2669] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:53.575414 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.587078 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a44ad-0a2d-4b74-9b19-ce4dd81b3eb7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.591731 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cd43d-382b-4dc9-a000-346405144936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.592366 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd43d-382b-4dc9-a000-346405144936 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.592646 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd43d-382b-4dc9-a000-346405144936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.592917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd43d-382b-4dc9-a000-346405144936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.593558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cd43d-382b-4dc9-a000-346405144936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.595732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cd43d-382b-4dc9-a000-346405144936 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.609361 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1334/2670] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:28:53.614204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.624481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43d325-e868-46a3-84bc-c6caa114198b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.631103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.631785 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.632055 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.632503 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] PATH_INFO: `/v3/projects/a8817015e8b14781acd7168f92ed7a6b/users/1a09ecfc95494341bc81e5d32c4bec42/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.652289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8817015e8b14781acd7168f92ed7a6b, user_id=1a09ecfc95494341bc81e5d32c4bec42, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.655166 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.673280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1337/2671] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:28:53 2022] PUT /identity/v3/projects/a8817015e8b14781acd7168f92ed7a6b/users/1a09ecfc95494341bc81e5d32c4bec42/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:28:53.684783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.685072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.685351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.715614 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] MFA Rules not processed for user `1a09ecfc95494341bc81e5d32c4bec42`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:28:53.716917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:53.787124 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1335/2672] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:28:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:28:53.799029 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:28:53.800902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:28:53.864403 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f9e15-c56c-483c-8a14-9a4605466c2e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:28:53.868128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:28:53.868779 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:28:53.869116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:28:53.869340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:28:53.878913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:28:53.880390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:28:53.889688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1338/2673] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:28:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:03.783011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:03.885500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:12.943262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:12.944741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:13.022691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e206e3-bb8c-464e-8226-a9466e1e3467 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:13.027162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:13.027694 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:13.027970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:13.028280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:13.030103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:13.043639 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:13.045294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29010229-589d-4148-867f-adf2bd258ab1 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:13.056393 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1336/2674] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:14.993841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:14.995632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:15.078901 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658c9d5-7cbc-47ab-ab51-1799294e460a service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:15.084217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:15.085005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.085291 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.085565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] PATH_INFO: `/v3/users/ddfebf61632d4efdb04269f7c9e2f654` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.086721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfebf61632d4efdb04269f7c9e2f654)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:15.088699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:15.143772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfebf61632d4efdb04269f7c9e2f654'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.159879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfebf61632d4efdb04269f7c9e2f654'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.170741 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfebf61632d4efdb04269f7c9e2f654'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.171342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfebf61632d4efdb04269f7c9e2f654'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.171934 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfebf61632d4efdb04269f7c9e2f654'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.187324 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1339/2675] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:29:14 2022] DELETE /identity/v3/users/ddfebf61632d4efdb04269f7c9e2f654 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:29:15.192758 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.193442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.193442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.213060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:15.214270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:15.282542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1337/2676] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:15.292712 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:15.294153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:15.355264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134114e8-8ec5-40e8-88d1-5bd3ae5c1248 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:15.359278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bf7da-ecd4-456e-967f-520662ee07bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:15.359920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf7da-ecd4-456e-967f-520662ee07bb service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.360216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf7da-ecd4-456e-967f-520662ee07bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.360498 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf7da-ecd4-456e-967f-520662ee07bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.371700 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf7da-ecd4-456e-967f-520662ee07bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:15.373197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf7da-ecd4-456e-967f-520662ee07bb service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:15.384044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1340/2677] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:15.685105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:15.686644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:15.704532 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc08c9-06a9-43ad-8622-baff8dbebd8e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:15.718867 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:15.719475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.719753 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.720053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] PATH_INFO: `/v3/projects/34db09cf16f14a49b4d87b4be5aafdb6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.722497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34db09cf16f14a49b4d87b4be5aafdb6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:15.724449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:15.789435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34db09cf16f14a49b4d87b4be5aafdb6', 'request_id': 'req-5a3e8137-a0aa-434c-a787-933ed04c3017'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.795285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34db09cf16f14a49b4d87b4be5aafdb6', 'request_id': 'req-5a3e8137-a0aa-434c-a787-933ed04c3017'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.802071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34db09cf16f14a49b4d87b4be5aafdb6', 'request_id': 'req-5a3e8137-a0aa-434c-a787-933ed04c3017'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.817497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] The token cache is being invalidate because project 34db09cf16f14a49b4d87b4be5aafdb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:29:15.817986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.822148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5a3e8137-a0aa-434c-a787-933ed04c3017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:15.823817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1338/2678] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:29:15 2022] DELETE /identity/v3/projects/34db09cf16f14a49b4d87b4be5aafdb6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:29:15.828793 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.829082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.829358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.849711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:15.850939 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:15.870149 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:15.916945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1341/2679] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:15.924623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.924940 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.925224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.945786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:15.947022 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:15.956993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1339/2680] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:15.961371 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:15.971788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192402ce-5344-4e69-8f08-a24325b2e04d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:15.976871 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:15.977488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:15.977838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:15.978113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:15.978768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:29:15.979077 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:15.980728 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:15.999486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1342/2681] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:29:16.003335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:16.013934 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fa99c-1304-4c66-b4bf-8a1338b417e8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:16.018455 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:16.019071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.019347 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.019620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.020291 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:16.022331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:16.027373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:16.060432 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1340/2682] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:29:16 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:29:16.063484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:16.074638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d46f-55c5-4e06-bba5-e44f029ecbee admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:16.151588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:16.152367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.152718 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.153619 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.154356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:16.156875 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:16.184828 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1343/2683] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:29:16 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:29:16.186282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:16.198751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6dcee-0e69-4764-9120-2800813907b4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:16.204655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:16.205864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.206146 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.206419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.207069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:16.208791 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:16.220053 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1341/2684] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:29:16.225203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:16.240801 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e2c36-4a65-44d0-b146-b6de1ca65fbb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:16.246219 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c30b2-c7fb-4dff-958b-c885eae3d9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:16.246845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c30b2-c7fb-4dff-958b-c885eae3d9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.247146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c30b2-c7fb-4dff-958b-c885eae3d9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.247396 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c30b2-c7fb-4dff-958b-c885eae3d9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.248038 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c30b2-c7fb-4dff-958b-c885eae3d9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:16.249679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c30b2-c7fb-4dff-958b-c885eae3d9b8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:16.271349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1344/2685] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:29:16.276048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:16.293550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f70627-3a52-45fd-b281-eb95f32edc91 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:16.298374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:16.299083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.299367 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.299654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] PATH_INFO: `/v3/projects/f205a572baf449f09e403e3750ee8af2/users/b467d4010a5844e3abcb2736e9c13716/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.315956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f205a572baf449f09e403e3750ee8af2, user_id=b467d4010a5844e3abcb2736e9c13716, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:16.318599 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:16.338297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1342/2686] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:29:16 2022] PUT /identity/v3/projects/f205a572baf449f09e403e3750ee8af2/users/b467d4010a5844e3abcb2736e9c13716/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:29:16.343994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d9292-c5a7-41b0-b92a-53bd0c2e3761 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.344314 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d9292-c5a7-41b0-b92a-53bd0c2e3761 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.344614 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d9292-c5a7-41b0-b92a-53bd0c2e3761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.366141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0c8d9292-c5a7-41b0-b92a-53bd0c2e3761 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:16.367384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0c8d9292-c5a7-41b0-b92a-53bd0c2e3761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:16.438435 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1345/2687] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:29:16.448627 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:16.450477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:16.526921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90850162-fe11-474d-8a37-19f2b9058e8b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:16.530742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:16.531656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:16.531943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:16.532261 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:16.543817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:16.545371 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:16.556399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1343/2688] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:17.261971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13df96-0482-4e8f-a4e4-9f91ceedcef7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:17.262526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13df96-0482-4e8f-a4e4-9f91ceedcef7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:17.262693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13df96-0482-4e8f-a4e4-9f91ceedcef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:17.300791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-db13df96-0482-4e8f-a4e4-9f91ceedcef7 None None] MFA Rules not processed for user `b467d4010a5844e3abcb2736e9c13716`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:17.302064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-db13df96-0482-4e8f-a4e4-9f91ceedcef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:17.321524 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:17.384740 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1346/2689] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:17.400755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:17.402357 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:17.477905 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7ebc9-f59e-4bbc-8c45-f33707971aba service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:17.482501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:17.483112 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:17.483387 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:17.483655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:17.493892 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:17.495380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:17.505394 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1344/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:18.916178 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:18.916758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:18.916758 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:18.938952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] MFA Rules not processed for user `b467d4010a5844e3abcb2736e9c13716`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:18.941282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:18.958643 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1347/2691] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:19.579687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:19.581492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:19.650202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:19.668670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d6e7-416c-4fff-9d97-a3296ee279e0 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:19.674566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:19.675275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:19.675559 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:19.675831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:19.677795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:19.690283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:19.691943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b37d3-c31b-4381-bc93-67182fd9e519 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:19.702676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1345/2692] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:28.933365 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=22 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:29.698248 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:30.811426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:30.813216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:30.905637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e241179-0187-42d5-9977-3dc864d6dcae None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:30.913513 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:30.914516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:30.914968 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:30.915431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:30.917775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:30.933593 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:30.935820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:30.947104 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1348/2693] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:37.960077 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2859f2-aab0-40de-b954-81236e983c7b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:37.961109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2859f2-aab0-40de-b954-81236e983c7b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:37.961109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2859f2-aab0-40de-b954-81236e983c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:38.111121 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8f2859f2-aab0-40de-b954-81236e983c7b None None] MFA Rules not processed for user `b467d4010a5844e3abcb2736e9c13716`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:38.112403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f2859f2-aab0-40de-b954-81236e983c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:38.119446 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1346/2694] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:29:40.942328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=34 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:41.200852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:41.202559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:41.216385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc769ae-bf1e-4544-8afb-373e11229596 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:41.223596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:41.223596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:41.224034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:41.224034 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] PATH_INFO: `/v3/users/1a09ecfc95494341bc81e5d32c4bec42` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:41.225380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a09ecfc95494341bc81e5d32c4bec42)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:41.227321 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:41.442444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a09ecfc95494341bc81e5d32c4bec42'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:41.461485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a09ecfc95494341bc81e5d32c4bec42'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:41.470618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a09ecfc95494341bc81e5d32c4bec42'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:41.471387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a09ecfc95494341bc81e5d32c4bec42'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:41.474177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a09ecfc95494341bc81e5d32c4bec42'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:41.488819 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1349/2695] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:29:41 2022] DELETE /identity/v3/users/1a09ecfc95494341bc81e5d32c4bec42 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:29:41.495424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:41.495979 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:41.496351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:41.519678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:41.520173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:41.589486 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1347/2696] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:41.599852 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:41.601653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:41.667721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeadc0-4dde-4fe3-98e3-477b16d731a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:41.671651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5a9b9-7d4a-4b10-82f7-2d3b5b1fed08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:41.672302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5a9b9-7d4a-4b10-82f7-2d3b5b1fed08 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:41.672580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5a9b9-7d4a-4b10-82f7-2d3b5b1fed08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:41.672845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5a9b9-7d4a-4b10-82f7-2d3b5b1fed08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:41.683982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5a9b9-7d4a-4b10-82f7-2d3b5b1fed08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:41.685444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5a9b9-7d4a-4b10-82f7-2d3b5b1fed08 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:41.696585 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1350/2697] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:29:41.983708 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:41.985659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:42.002199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ef1e-7630-4b6b-bf07-676fd55f6ecb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.008698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.010004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.010275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.010549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] PATH_INFO: `/v3/projects/a8817015e8b14781acd7168f92ed7a6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.011913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8817015e8b14781acd7168f92ed7a6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.013868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.071149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8817015e8b14781acd7168f92ed7a6b', 'request_id': 'req-872f57c8-c51c-4c89-8b55-3956448d5864'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:42.079385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8817015e8b14781acd7168f92ed7a6b', 'request_id': 'req-872f57c8-c51c-4c89-8b55-3956448d5864'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:42.086798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8817015e8b14781acd7168f92ed7a6b', 'request_id': 'req-872f57c8-c51c-4c89-8b55-3956448d5864'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:42.098491 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] The token cache is being invalidate because project a8817015e8b14781acd7168f92ed7a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:29:42.098766 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:42.099445 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-872f57c8-c51c-4c89-8b55-3956448d5864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:29:42.100986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1348/2698] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:29:41 2022] DELETE /identity/v3/projects/a8817015e8b14781acd7168f92ed7a6b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:29:42.105940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.106472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.106472 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.126590 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:42.127834 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:42.194663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1351/2699] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:42.261148 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.261430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.261700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.281898 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:42.283144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:42.292080 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1349/2700] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:42.296507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.307223 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8181a81-a2a3-4cd0-8011-b9e6e4a1acb8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.312772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.313516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.313802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.314079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.314727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:29:42.315185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.317094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.329872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1352/2701] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:29:42.334661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.344251 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b59b9-c012-457e-b32a-9d2dbe531c88 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.349697 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.350310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.350584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.350851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.351471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.353638 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.383654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1350/2702] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:29:42 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:29:42.387976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.393883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:42.401603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284bd98-d33a-4d47-bb4b-a545e436c744 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.467922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.468547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.468709 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.468981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.469636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.472103 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.493773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1353/2703] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:29:42 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:29:42.497978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.509141 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b049f-52fd-4f0b-b14a-fd2dd1a7e656 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.513966 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.514579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.514857 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.515130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.515776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.517429 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.528670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1351/2704] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:29:42.533222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.544843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a745eb-3cf2-4350-93ba-34fe79bdc39d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.550520 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3e6e1-01cc-4614-9750-fabfc03af344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.551128 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3e6e1-01cc-4614-9750-fabfc03af344 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.551402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3e6e1-01cc-4614-9750-fabfc03af344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.551668 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3e6e1-01cc-4614-9750-fabfc03af344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.552349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3e6e1-01cc-4614-9750-fabfc03af344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.553933 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3e6e1-01cc-4614-9750-fabfc03af344 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.564457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1354/2705] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:29:42.569401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.579650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f43c7-b675-405b-83ec-e09b69f9bae7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.585360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.586143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.586442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.586717 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] PATH_INFO: `/v3/projects/5c4565c8330b46329441ef589a556ca5/users/961ba15ee0404cf2a66c2c26a01c96ed/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.604993 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4565c8330b46329441ef589a556ca5, user_id=961ba15ee0404cf2a66c2c26a01c96ed, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.607702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.622768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1352/2706] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:29:42 2022] PUT /identity/v3/projects/5c4565c8330b46329441ef589a556ca5/users/961ba15ee0404cf2a66c2c26a01c96ed/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:29:42.628014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068087b4-7e29-4319-8996-4771317fc009 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.628328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068087b4-7e29-4319-8996-4771317fc009 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.628635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068087b4-7e29-4319-8996-4771317fc009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.650904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-068087b4-7e29-4319-8996-4771317fc009 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:42.652122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-068087b4-7e29-4319-8996-4771317fc009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:42.718228 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1355/2707] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:29:42.728844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:42.730517 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:42.745528 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:42.794713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5da8-48c3-4112-a9ba-ae44a952969c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:42.798369 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:42.798976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:42.799270 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:42.800831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:42.809919 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:42.811512 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:42.822495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1353/2708] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:29:43.520635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649c48-9c75-4ab9-a7a0-376a9b00ea3e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:43.521239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649c48-9c75-4ab9-a7a0-376a9b00ea3e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:43.521357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649c48-9c75-4ab9-a7a0-376a9b00ea3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:43.559991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a0649c48-9c75-4ab9-a7a0-376a9b00ea3e None None] MFA Rules not processed for user `961ba15ee0404cf2a66c2c26a01c96ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:43.561282 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a0649c48-9c75-4ab9-a7a0-376a9b00ea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:43.637870 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1356/2709] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:43.652757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:29:43.654835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:43.726107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5a4a0-a79e-48ed-803f-abfc76e7da5f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:29:43.726107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c93f-3aa7-466b-ad3d-c0542444f065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:29:43.726107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c93f-3aa7-466b-ad3d-c0542444f065 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:43.726107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c93f-3aa7-466b-ad3d-c0542444f065 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:43.726629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c93f-3aa7-466b-ad3d-c0542444f065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:43.735448 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700c93f-3aa7-466b-ad3d-c0542444f065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:29:43.737000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700c93f-3aa7-466b-ad3d-c0542444f065 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:29:43.746772 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1354/2710] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:29:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:29:45.122210 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:29:45.122745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:29:45.122745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:29:45.145556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] MFA Rules not processed for user `961ba15ee0404cf2a66c2c26a01c96ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:29:45.146810 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:29:45.153686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1357/2711] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:29:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:29:53.742165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=46 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:29:55.139265 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:02.560615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:02.561252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:02.561252 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:02.580798 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:02.582014 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:02.642818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1355/2712] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Aug 18 11:30:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:02.697606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:02.699017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:02.785347 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669edac-329e-4e0a-89cd-d5c3425f9de8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:02.790375 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0f212-4717-448f-a064-44aa36ab8ef6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:02.790990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0f212-4717-448f-a064-44aa36ab8ef6 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:02.791264 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0f212-4717-448f-a064-44aa36ab8ef6 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:02.791531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0f212-4717-448f-a064-44aa36ab8ef6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:02.801221 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0f212-4717-448f-a064-44aa36ab8ef6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:02.802830 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0f212-4717-448f-a064-44aa36ab8ef6 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:02.811529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1358/2713] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:30:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:30:12.633188 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:12.808014 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=37 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:13.074480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:13.086144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa043393-7c84-4cc6-acea-dd9b0f5cd827 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:13.089880 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:13.090497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:13.090819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:13.091043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:13.092874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:13.170060 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:13.171543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:13.181741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1356/2714] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:30:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:30:22.257469 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070239e3-3cfc-4c83-ac99-e54da0cbfa49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:22.258106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070239e3-3cfc-4c83-ac99-e54da0cbfa49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:22.258317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070239e3-3cfc-4c83-ac99-e54da0cbfa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:22.283863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-070239e3-3cfc-4c83-ac99-e54da0cbfa49 None None] MFA Rules not processed for user `c59d7dff52ce4e1f83039f3fe2288e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:22.285732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-070239e3-3cfc-4c83-ac99-e54da0cbfa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:22.293927 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1359/2715] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:30:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:22.730606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:22.731143 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:22.749249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090577-caa1-494a-bc0d-55fbe8d20e7c service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:22.753547 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:22.754373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:22.754814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:22.755249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:22.761679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:22.773354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:22.774860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:22.789055 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1357/2716] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:30:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:30:23.876581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:23.877803 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:23.877900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:23.896872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] MFA Rules not processed for user `25f03b43ee624965b53e9e3163fbeb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:23.898088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:23.905207 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1360/2717] 10.222.0.16 () {58 vars in 967 bytes} [Thu Aug 18 11:30:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:23.917481 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:23.930167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9862ca0-d149-4de1-8eef-929443600ac9 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:23.934956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ca19cdbc204ba995b2d8d46a914fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:23.935536 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:23.935836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:23.936103 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:23.944663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:23.946222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:23.953959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1358/2718] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Aug 18 11:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 11:30:26.312842 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:26.314486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:26.413440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2329e2-4a3b-43b6-b946-d4cd95b8d22e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:26.418463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:26.420627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:26.421521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:26.422082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:26.422557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:26.423501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:26.425245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:26.444321 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f696-79d5-44aa-9b62-e26736003392 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:26.448726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:26.452112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:26.452561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:26.453440 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:26.453884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:26.454324 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] PATH_INFO: `/v3/users/b467d4010a5844e3abcb2736e9c13716` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:26.456426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b467d4010a5844e3abcb2736e9c13716)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:26.459477 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:26.468548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1361/2719] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:30:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:30:26.534821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b467d4010a5844e3abcb2736e9c13716'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.535901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b467d4010a5844e3abcb2736e9c13716'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.538370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b467d4010a5844e3abcb2736e9c13716'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.563874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b467d4010a5844e3abcb2736e9c13716'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.581632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-09191480-0cb3-450c-8dc5-165ac79cec90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b467d4010a5844e3abcb2736e9c13716'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.598653 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1359/2720] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:30:26 2022] DELETE /identity/v3/users/b467d4010a5844e3abcb2736e9c13716 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:30:26.794778 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:26.806691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ca300-c6e1-4ed7-b625-c7087e3fd9db service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:26.872557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:26.873287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:26.873711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:26.873991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] PATH_INFO: `/v3/projects/f205a572baf449f09e403e3750ee8af2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:26.875130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f205a572baf449f09e403e3750ee8af2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:26.877737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:26.928788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f205a572baf449f09e403e3750ee8af2', 'request_id': 'req-2199ede5-b84e-4dcd-91b1-a016f1366f07'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.935346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f205a572baf449f09e403e3750ee8af2', 'request_id': 'req-2199ede5-b84e-4dcd-91b1-a016f1366f07'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.944456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f205a572baf449f09e403e3750ee8af2', 'request_id': 'req-2199ede5-b84e-4dcd-91b1-a016f1366f07'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.960938 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] The token cache is being invalidate because project f205a572baf449f09e403e3750ee8af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:30:26.961490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.962577 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2199ede5-b84e-4dcd-91b1-a016f1366f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:26.968774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1362/2721] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:30:26 2022] DELETE /identity/v3/projects/f205a572baf449f09e403e3750ee8af2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:30:26.971511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:26.971795 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:26.972134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:26.992814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:26.994136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:27.047426 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:27.078642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1360/2722] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:27.146236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.146514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.146784 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.166748 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:27.173227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:27.177721 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:27.189007 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1363/2723] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:27.195358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.205611 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe77e8-cd29-4d18-869b-9519a34b3029 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.212632 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.213475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.213913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.214297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.215087 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:30:27.215593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.218117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.239989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1361/2724] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:30:27.241430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.251376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967782f7-a4e1-4e2c-85c5-e05ae227188d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.258335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.259569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.259850 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.260234 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.261129 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.263407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.301445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1364/2725] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:30:27 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:30:27.306468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.332311 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1f341-6cd2-4bb6-b2c5-04658da6d017 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.421179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.421886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.422076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.422487 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.423191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.425903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.449161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1362/2726] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:30:27 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:30:27.453412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.465693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c06ef-ccb6-4072-b91c-df41b4c9aebe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.470372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.471840 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.472113 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.472415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.473063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.476818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.486955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1365/2727] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:30:27.491214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.501695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b4e97-c922-4061-b738-7fcb3187c26b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.508926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf945f3-c11a-4296-8789-3e981c577e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.509548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf945f3-c11a-4296-8789-3e981c577e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.509830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf945f3-c11a-4296-8789-3e981c577e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.510100 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf945f3-c11a-4296-8789-3e981c577e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.510751 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf945f3-c11a-4296-8789-3e981c577e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.512402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf945f3-c11a-4296-8789-3e981c577e94 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.523754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1363/2728] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:30:27.528239 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.539130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c87502-a15d-43e9-a568-948b90344cb8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.543950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.544649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.544926 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.545195 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] PATH_INFO: `/v3/projects/7da7bc41a79f495aab8c0708f8b20def/users/498d3ea763e14039b32d7888e5019fe1/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.562447 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da7bc41a79f495aab8c0708f8b20def, user_id=498d3ea763e14039b32d7888e5019fe1, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.566186 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.581559 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1366/2729] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:30:27 2022] PUT /identity/v3/projects/7da7bc41a79f495aab8c0708f8b20def/users/498d3ea763e14039b32d7888e5019fe1/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:30:27.591631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.591918 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.592218 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.628519 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] MFA Rules not processed for user `498d3ea763e14039b32d7888e5019fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:27.630325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:27.703952 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1364/2730] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:30:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:27.717431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.724167 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:27.785158 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2362c-b2b6-4e12-abc9-36f49c25a4f0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.789455 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a132379-b0cc-48aa-b756-22c6de1d28ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.790046 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132379-b0cc-48aa-b756-22c6de1d28ea service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.790326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132379-b0cc-48aa-b756-22c6de1d28ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.790597 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132379-b0cc-48aa-b756-22c6de1d28ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.806516 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a132379-b0cc-48aa-b756-22c6de1d28ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.808026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a132379-b0cc-48aa-b756-22c6de1d28ea service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.818230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1367/2731] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:30:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:30:27.844732 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:27.891721 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:27.914444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f14a8-56df-4d0f-95ad-241ffd6a07f5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:27.918989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:27.919601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:27.919874 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:27.920142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] PATH_INFO: `/v3/users/498d3ea763e14039b32d7888e5019fe1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:27.921373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498d3ea763e14039b32d7888e5019fe1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:27.923280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:27.974101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498d3ea763e14039b32d7888e5019fe1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:27.974782 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498d3ea763e14039b32d7888e5019fe1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:27.975399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498d3ea763e14039b32d7888e5019fe1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:27.987657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498d3ea763e14039b32d7888e5019fe1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:27.994903 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498d3ea763e14039b32d7888e5019fe1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:28.007901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1365/2732] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:30:27 2022] DELETE /identity/v3/users/498d3ea763e14039b32d7888e5019fe1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:30:28.012332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:28.012627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:28.012904 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:28.039446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:30:28.040694 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:28.104579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1368/2733] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:30:28.115336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:28.116810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:28.187764 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e7036-2dd3-4adc-aab2-028522ddbdb3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:28.193380 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:28.194045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:28.194364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:28.194636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:28.204551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:28.206310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:28.219588 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1366/2734] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:30:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:30:28.533483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:28.535286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:28.547707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282e6df-2c70-48f6-90fb-3095a364f2e7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:28.553686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:28.554440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:28.554900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:28.555170 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] PATH_INFO: `/v3/projects/7da7bc41a79f495aab8c0708f8b20def` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:28.556343 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da7bc41a79f495aab8c0708f8b20def)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:28.558242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:28.608471 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:28.616218 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da7bc41a79f495aab8c0708f8b20def', 'request_id': 'req-694310de-4e46-40af-bb76-3532555d8053'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:28.622386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da7bc41a79f495aab8c0708f8b20def', 'request_id': 'req-694310de-4e46-40af-bb76-3532555d8053'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:28.630437 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da7bc41a79f495aab8c0708f8b20def', 'request_id': 'req-694310de-4e46-40af-bb76-3532555d8053'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:28.642713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] The token cache is being invalidate because project 7da7bc41a79f495aab8c0708f8b20def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:30:28.643068 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:28.643743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:30:28.645456 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1369/2735] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:30:28 2022] DELETE /identity/v3/projects/7da7bc41a79f495aab8c0708f8b20def => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:30:38.212260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:38.639330 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:30:58.793520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:30:58.803557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614b357-567d-482c-9ed3-fae7df07d465 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:30:58.855439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:30:58.856040 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:30:58.856353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:30:58.856622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:30:58.858467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:30:58.939360 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:30:58.941179 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895bc4d-f78f-4984-a82c-aef17b178dd2 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:30:58.953176 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1367/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:30:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:31:00.146960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:31:00.148595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:31:00.161836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694310de-4e46-40af-bb76-3532555d8053 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:31:00.165410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:31:00.166022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:31:00.166289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:31:00.166550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:31:00.168269 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:31:00.183420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:31:00.184886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:31:00.196567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1370/2737] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:31:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:31:08.947561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=54 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:31:10.191702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:31:13.526962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:31:13.527581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:31:13.527581 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:31:13.551306 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] MFA Rules not processed for user `961ba15ee0404cf2a66c2c26a01c96ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:31:13.553154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:31:13.619459 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1368/2738] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:31:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:31:13.904604 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:31:13.913786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f60619-900f-478c-a761-acc0c9c447c1 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:31:13.917473 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:31:13.918069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:31:13.918342 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:31:13.918609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:31:13.926065 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:31:13.938407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:31:13.939883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:31:13.950222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1371/2739] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:31:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:31:23.612113 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:31:23.946102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:32:15.507167 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:15.516879 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aee775-c212-4878-9f51-5842ccb69e01 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:15.521353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:15.521944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:15.522223 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:15.522492 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:15.524203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:15.534661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:15.536104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:15.544409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1369/2740] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:32:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:32:18.019991 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.021548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:18.090891 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff41b1-2b58-461e-8dcf-b67cff56f541 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.095380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.096035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.096354 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.096629 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] PATH_INFO: `/v3/users/961ba15ee0404cf2a66c2c26a01c96ed` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.098183 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961ba15ee0404cf2a66c2c26a01c96ed)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.100075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.148978 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961ba15ee0404cf2a66c2c26a01c96ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.161854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961ba15ee0404cf2a66c2c26a01c96ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.169674 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961ba15ee0404cf2a66c2c26a01c96ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.170293 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961ba15ee0404cf2a66c2c26a01c96ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.170918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-25f9a57d-c15d-4993-b746-c56365ebe287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961ba15ee0404cf2a66c2c26a01c96ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.185416 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1372/2741] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:32:18 2022] DELETE /identity/v3/users/961ba15ee0404cf2a66c2c26a01c96ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:32:18.375385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.389355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7f6d0-1f79-4aaa-bd71-73674ce2cf22 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.451278 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.451843 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.452120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.452421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] PATH_INFO: `/v3/projects/5c4565c8330b46329441ef589a556ca5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.453549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4565c8330b46329441ef589a556ca5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.455447 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.507019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4565c8330b46329441ef589a556ca5', 'request_id': 'req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.514971 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4565c8330b46329441ef589a556ca5', 'request_id': 'req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.521439 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4565c8330b46329441ef589a556ca5', 'request_id': 'req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.536757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] The token cache is being invalidate because project 5c4565c8330b46329441ef589a556ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:32:18.537048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.537726 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-463bf7d7-2091-4e71-96b8-9ff3f7a45c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:18.539242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1370/2742] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:32:18 2022] DELETE /identity/v3/projects/5c4565c8330b46329441ef589a556ca5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:32:18.544085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.544417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.544693 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.564945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:32:18.566986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:18.625564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1373/2743] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:32:18.633973 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.634259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.634533 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.654449 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:32:18.655670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:18.664603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1371/2744] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:32:18.668911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.679268 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7aa9d3-a428-4e61-8fdb-551f3ade788b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.683832 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.684464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.684747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.685019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.685661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:32:18.685963 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.687544 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.699114 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1374/2745] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:32:18.703666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.713290 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc3970-17af-4dc1-9908-b1a7a03e5777 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.717942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.718557 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.718833 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.719102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.719723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.722208 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.751634 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1372/2746] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:32:18 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:32:18.756054 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.767987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12ee0-35ca-4cd3-ad1f-ef1e4efa2883 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.774378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:32:18.830918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.831682 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.831955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.832245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.832856 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.834746 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.855710 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1375/2747] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:32:18 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:32:18.859886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.870851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcd497-2a68-41d8-a679-7fb4fe2a28f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.875475 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.876073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.876386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.876656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.877298 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.879607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.890044 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1373/2748] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:32:18.894745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.904713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1110fa-8634-437d-b9c5-037a47cc1869 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.909631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00623ddc-fb8e-4453-b212-16e82009bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.910248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00623ddc-fb8e-4453-b212-16e82009bda3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.910528 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00623ddc-fb8e-4453-b212-16e82009bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.910797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00623ddc-fb8e-4453-b212-16e82009bda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.911440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00623ddc-fb8e-4453-b212-16e82009bda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.913074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00623ddc-fb8e-4453-b212-16e82009bda3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.924070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1376/2749] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:32:18.928587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:18.939000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e0e86-6bc9-4360-a047-5fdfb7f97c74 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:18.944374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:18.945027 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.945308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.945583 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] PATH_INFO: `/v3/projects/221b42f866f4499e808d2108205a6a79/users/3f03e1ba18a94a0ca10d35c3dbed178e/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:18.961091 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221b42f866f4499e808d2108205a6a79, user_id=3f03e1ba18a94a0ca10d35c3dbed178e, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:18.963679 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:18.984786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1374/2750] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:32:18 2022] PUT /identity/v3/projects/221b42f866f4499e808d2108205a6a79/users/3f03e1ba18a94a0ca10d35c3dbed178e/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:32:18.996576 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:18.996857 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:18.997130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:19.027005 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] MFA Rules not processed for user `3f03e1ba18a94a0ca10d35c3dbed178e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:32:19.028638 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:19.096676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1377/2751] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:32:19.111019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:19.113586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:19.128418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:32:19.172501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032e3e-33cf-4905-850e-3f75b5a840d7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:19.177067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7f0df-29e3-410c-8d5b-271e1443cc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:19.177671 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7f0df-29e3-410c-8d5b-271e1443cc0c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:19.177947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7f0df-29e3-410c-8d5b-271e1443cc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:19.178214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7f0df-29e3-410c-8d5b-271e1443cc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:19.186640 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7f0df-29e3-410c-8d5b-271e1443cc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:19.188052 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7f0df-29e3-410c-8d5b-271e1443cc0c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:19.197776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1375/2752] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:32:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:32:27.315649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:27.378532 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492e3d7-9fce-4858-b5c3-e0580ba44da5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:27.383061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:27.383690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:27.383962 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:27.384255 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] PATH_INFO: `/v3/users/3f03e1ba18a94a0ca10d35c3dbed178e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:27.385411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f03e1ba18a94a0ca10d35c3dbed178e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:27.387295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:27.406911 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:32:27.439866 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f03e1ba18a94a0ca10d35c3dbed178e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:27.452013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f03e1ba18a94a0ca10d35c3dbed178e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:27.459837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f03e1ba18a94a0ca10d35c3dbed178e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:27.460474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f03e1ba18a94a0ca10d35c3dbed178e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:27.461082 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f03e1ba18a94a0ca10d35c3dbed178e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:27.474557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1378/2753] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:32:27 2022] DELETE /identity/v3/users/3f03e1ba18a94a0ca10d35c3dbed178e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:32:27.479109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:27.479622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:27.479723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:27.498336 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:32:27.499542 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:27.565325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1376/2754] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:32:27.576093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:27.577595 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:27.637366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2910f7-500b-4e24-b2fa-b5e6ed1746c2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:27.641096 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:27.641757 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:27.641960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:27.642235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:27.651822 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:27.653327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:27.664490 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1379/2755] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 11:32:27.952289 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:32:27.953745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:32:27.969351 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8bc3e-8bdd-4341-a802-4a0766d0cd5e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:32:27.974282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:32:27.975665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:32:27.975948 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:32:27.976418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] PATH_INFO: `/v3/projects/221b42f866f4499e808d2108205a6a79` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:32:27.977601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221b42f866f4499e808d2108205a6a79)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:32:27.979510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:32:28.041994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221b42f866f4499e808d2108205a6a79', 'request_id': 'req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:28.052520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221b42f866f4499e808d2108205a6a79', 'request_id': 'req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:28.059165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221b42f866f4499e808d2108205a6a79', 'request_id': 'req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:28.070442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] The token cache is being invalidate because project 221b42f866f4499e808d2108205a6a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:32:28.070663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:28.071348 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:32:28.072869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1377/2756] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:32:27 2022] DELETE /identity/v3/projects/221b42f866f4499e808d2108205a6a79 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:32:37.659412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:32:38.067266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:01.803787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:01.805287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:01.903859 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1fbba-9bfd-4d89-bb9a-ab580d337529 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:01.977952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:01.978922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:01.979950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:01.980609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:01.982390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:01.996441 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:01.997967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:02.011818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1380/2757] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:07.475652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:07.477421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:07.534682 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8c9c1-ec13-46ea-993e-f9eec03e5906 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:07.538406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:07.539000 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:07.539286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:07.539556 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:07.541308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:07.606239 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:07.607667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:07.617822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1378/2758] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:10.668156 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:10.669764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:10.748161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:10.751191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:10.770526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ca03c-158a-401b-81b1-11837dd42ade service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:10.773887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a8fa-de3f-4cd4-a9d3-ece231e6e493 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:10.777200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:10.778157 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:10.778618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:10.779074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:10.783809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:10.784024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:10.784024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:10.784024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:10.784024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] PATH_INFO: `/v3/users/c59d7dff52ce4e1f83039f3fe2288e6a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:10.784900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59d7dff52ce4e1f83039f3fe2288e6a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:10.786941 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:10.795400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:10.796994 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:10.807459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1381/2759] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:10.842111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59d7dff52ce4e1f83039f3fe2288e6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:10.842775 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59d7dff52ce4e1f83039f3fe2288e6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:10.843420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59d7dff52ce4e1f83039f3fe2288e6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:10.858392 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59d7dff52ce4e1f83039f3fe2288e6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:10.869618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4af95ac7-b1d5-4316-88ab-81cf2bcc1581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59d7dff52ce4e1f83039f3fe2288e6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:10.882804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1379/2760] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:10 2022] DELETE /identity/v3/users/c59d7dff52ce4e1f83039f3fe2288e6a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:33:11.058457 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.071452 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c924c30-525a-47b9-ab87-867d329c488c service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.081026 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:11.134411 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.134982 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.135261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.135531 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] PATH_INFO: `/v3/projects/28a8db4d53754a09aad85d3b756fed10` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.136689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28a8db4d53754a09aad85d3b756fed10)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.138578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.189692 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28a8db4d53754a09aad85d3b756fed10', 'request_id': 'req-3d6fa643-f79a-4e19-a062-0f4a7091b411'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:11.195816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28a8db4d53754a09aad85d3b756fed10', 'request_id': 'req-3d6fa643-f79a-4e19-a062-0f4a7091b411'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:11.203176 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28a8db4d53754a09aad85d3b756fed10', 'request_id': 'req-3d6fa643-f79a-4e19-a062-0f4a7091b411'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:11.214744 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] The token cache is being invalidate because project 28a8db4d53754a09aad85d3b756fed10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:11.215025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:11.215669 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d6fa643-f79a-4e19-a062-0f4a7091b411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:11.217185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1382/2761] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:11 2022] DELETE /identity/v3/projects/28a8db4d53754a09aad85d3b756fed10 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:11.222774 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.223073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.223417 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.244308 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:11.245504 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:11.262064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:11.308003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1380/2762] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:33:11.316717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.316999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.317267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.336943 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:11.338133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:11.346661 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1383/2763] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:11.350998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.360408 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c07cc-ae8c-4ba7-b4f1-868fc7da36b0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.365565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.366150 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.366421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.366687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.367318 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:33:11.367616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.369219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.381035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1381/2764] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:33:11.385394 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.395326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fc7d3-8de5-4c8a-b438-ca7590ad3cf7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.399955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.400578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.400855 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.401123 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.401735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.403625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.431358 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1384/2765] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:11.435495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.445716 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138deeb-d6e3-4a53-a013-cd51ce240dea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.501811 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.502405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.502683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.502956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.503822 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.505785 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.527094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1382/2766] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:11.531433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.542703 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02652a1-507c-4193-819c-da5d314c7317 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.547400 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.547990 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.548297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.548570 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.549204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.550817 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.560603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1385/2767] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:11.564622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.574810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781faad9-e399-439c-be75-8a281e2dc4df admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.579418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13ab31-f7c0-4177-9c19-71e473480fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.580003 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13ab31-f7c0-4177-9c19-71e473480fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.580316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13ab31-f7c0-4177-9c19-71e473480fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.580587 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13ab31-f7c0-4177-9c19-71e473480fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.581744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13ab31-f7c0-4177-9c19-71e473480fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.583409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13ab31-f7c0-4177-9c19-71e473480fa3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.593916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1383/2768] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:11.598443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.608341 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a848a-ac2a-423b-bdf0-b9bc3bb14811 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.613609 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.614242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.614521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.614802 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] PATH_INFO: `/v3/projects/884fe524215e49a988ec8368d75be24f/users/def8c6daf80e427ebfe9e6af27cde3e0/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.629920 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884fe524215e49a988ec8368d75be24f, user_id=def8c6daf80e427ebfe9e6af27cde3e0, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.632649 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.646160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1386/2769] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:11 2022] PUT /identity/v3/projects/884fe524215e49a988ec8368d75be24f/users/def8c6daf80e427ebfe9e6af27cde3e0/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:11.656582 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.656870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.657142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.685057 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] MFA Rules not processed for user `def8c6daf80e427ebfe9e6af27cde3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:11.686263 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:11.749830 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1384/2770] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:11.754502 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.796651 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:11.817485 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a174c-5098-47f6-859e-bfc7f11607b6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.822307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.822909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.823191 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.823465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.824079 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.826032 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.852773 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1387/2771] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:11.856997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.867460 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb383e3-a95c-494d-acae-33425439ed1a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.879884 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:11.930410 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.931071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.931331 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.931603 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.932262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.934243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.955616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1385/2772] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:11 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:11.960279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:11.970470 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124ef80-86d9-44a0-bee1-7ffb131d2466 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:11.975224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:11.975814 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:11.976093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:11.976405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:11.977091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:11.978689 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:11.989410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1388/2773] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:11.993485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:12.003949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b735e-a0c7-4b15-b8a4-33d91b967214 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:12.008696 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:12.009345 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.009623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.009900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] PATH_INFO: `/v3/projects/d6dc4a3701d54f80b110b9155f1fce36/users/10b7c4cfe615445c998e7ed359ac0129/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.025302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dc4a3701d54f80b110b9155f1fce36, user_id=10b7c4cfe615445c998e7ed359ac0129, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:12.027970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:12.042174 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1386/2774] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:11 2022] PUT /identity/v3/projects/d6dc4a3701d54f80b110b9155f1fce36/users/10b7c4cfe615445c998e7ed359ac0129/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:12.046179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:12.056214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101bafb-0351-4048-9d73-1c6a07c038aa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:12.112074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a165c2b-d77b-4d83-b79f-4e6a077b643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:12.112824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a165c2b-d77b-4d83-b79f-4e6a077b643a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.113106 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a165c2b-d77b-4d83-b79f-4e6a077b643a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.113379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a165c2b-d77b-4d83-b79f-4e6a077b643a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.114013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a165c2b-d77b-4d83-b79f-4e6a077b643a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:12.115596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a165c2b-d77b-4d83-b79f-4e6a077b643a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:12.125262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1389/2775] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:12.130065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:12.140502 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ac5bd-4d63-4877-97d3-f3b39e2e8826 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:12.145869 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:12.146514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.146792 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.147068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] PATH_INFO: `/v3/projects/d6dc4a3701d54f80b110b9155f1fce36/users/10b7c4cfe615445c998e7ed359ac0129/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.149011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dc4a3701d54f80b110b9155f1fce36, user_id=10b7c4cfe615445c998e7ed359ac0129, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:12.151633 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:12.162580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1387/2776] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:12 2022] PUT /identity/v3/projects/d6dc4a3701d54f80b110b9155f1fce36/users/10b7c4cfe615445c998e7ed359ac0129/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:12.174028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.174319 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.174599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.202381 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] MFA Rules not processed for user `10b7c4cfe615445c998e7ed359ac0129`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:12.203521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:12.271952 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1390/2777] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:12.283853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:12.285786 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:12.345768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a1a2-6769-4c4c-b24c-7d2ef884a827 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:12.349544 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07a8d3-7f3e-4d4d-867e-380c2f5b7c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:12.350142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07a8d3-7f3e-4d4d-867e-380c2f5b7c30 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.350438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07a8d3-7f3e-4d4d-867e-380c2f5b7c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.350687 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07a8d3-7f3e-4d4d-867e-380c2f5b7c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.359501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07a8d3-7f3e-4d4d-867e-380c2f5b7c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:12.360953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07a8d3-7f3e-4d4d-867e-380c2f5b7c30 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:12.371590 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1388/2778] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:12.787836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:12.825630 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:12.850545 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90602c51-9c41-4687-8c78-ec62ba3cabe1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:12.854967 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:12.855579 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.855853 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.856115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] PATH_INFO: `/v3/users/def8c6daf80e427ebfe9e6af27cde3e0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.857241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def8c6daf80e427ebfe9e6af27cde3e0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:12.859127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:12.903761 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def8c6daf80e427ebfe9e6af27cde3e0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:12.916015 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def8c6daf80e427ebfe9e6af27cde3e0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:12.922606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def8c6daf80e427ebfe9e6af27cde3e0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:12.923258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def8c6daf80e427ebfe9e6af27cde3e0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:12.923876 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def8c6daf80e427ebfe9e6af27cde3e0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:12.936356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1391/2779] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:12 2022] DELETE /identity/v3/users/def8c6daf80e427ebfe9e6af27cde3e0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:12.941744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:12.942326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:12.942326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:12.961085 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:12.962279 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:13.021472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1389/2780] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:13.031230 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:13.032666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:13.090009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d35476-0781-4d0a-b6cf-7844a1d69221 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:13.093686 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:13.094279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:13.094556 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:13.094825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:13.103985 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:13.105445 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:13.115646 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1392/2781] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:13.381910 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:13.383454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:13.397683 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1cd33-d562-4882-833d-d423d59ddaac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:13.402652 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:13.403283 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:13.403560 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:13.403829 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] PATH_INFO: `/v3/projects/884fe524215e49a988ec8368d75be24f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:13.404996 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=884fe524215e49a988ec8368d75be24f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:13.406913 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:13.416482 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:13.459354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884fe524215e49a988ec8368d75be24f', 'request_id': 'req-92911572-a5d9-4681-beb8-be513a3f0ced'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.465828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884fe524215e49a988ec8368d75be24f', 'request_id': 'req-92911572-a5d9-4681-beb8-be513a3f0ced'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.471926 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884fe524215e49a988ec8368d75be24f', 'request_id': 'req-92911572-a5d9-4681-beb8-be513a3f0ced'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.483069 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] The token cache is being invalidate because project 884fe524215e49a988ec8368d75be24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:13.483358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.483998 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.485523 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1390/2782] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:13 2022] DELETE /identity/v3/projects/884fe524215e49a988ec8368d75be24f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:13.490224 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:13.499659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80904acf-532b-4a60-82d3-557ab3f5f2b2 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:13.553788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:13.561387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:13.562212 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:13.562666 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:13.562997 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] PATH_INFO: `/v3/users/10b7c4cfe615445c998e7ed359ac0129` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:13.564421 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b7c4cfe615445c998e7ed359ac0129)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:13.567037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:13.611945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b7c4cfe615445c998e7ed359ac0129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.625249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b7c4cfe615445c998e7ed359ac0129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.633283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b7c4cfe615445c998e7ed359ac0129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.633913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b7c4cfe615445c998e7ed359ac0129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.634517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-64f33a4f-dff8-4e05-a0be-d034fa3e5c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b7c4cfe615445c998e7ed359ac0129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:13.646751 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1393/2783] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:13 2022] DELETE /identity/v3/users/10b7c4cfe615445c998e7ed359ac0129 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:13.915851 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:13.917386 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:13.931601 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92911572-a5d9-4681-beb8-be513a3f0ced admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:13.989404 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:13.990083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:13.990301 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:13.990574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] PATH_INFO: `/v3/projects/d6dc4a3701d54f80b110b9155f1fce36` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:13.991702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6dc4a3701d54f80b110b9155f1fce36)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:13.993656 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.042170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6dc4a3701d54f80b110b9155f1fce36', 'request_id': 'req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:14.049761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6dc4a3701d54f80b110b9155f1fce36', 'request_id': 'req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:14.056029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6dc4a3701d54f80b110b9155f1fce36', 'request_id': 'req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:14.066115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] The token cache is being invalidate because project d6dc4a3701d54f80b110b9155f1fce36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:14.066399 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:14.067119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f03d4608-0efb-4dbd-90f0-fa858e3b5ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:14.068609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1391/2784] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:13 2022] DELETE /identity/v3/projects/d6dc4a3701d54f80b110b9155f1fce36 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:14.074243 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.074743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.074851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.094831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:14.095965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:14.154726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1394/2785] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:14.163219 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.163511 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.163793 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.183065 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:14.184304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:14.192834 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1392/2786] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:14.196976 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.206538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a246c-2e45-4679-b17e-f5c481afae8c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.211900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.212518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.212791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.213057 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.213690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:33:14.213984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.215564 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.226953 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1395/2787] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:33:14.231180 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.240508 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6b578-cf11-4424-b778-b249888f5fa6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.245111 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.245698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.245976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.246243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.246854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.248804 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.269806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:14.278185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1393/2788] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:14.282340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.292816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878241e-0343-4ff9-8d04-deae40e5aae1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.348723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.349246 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.349526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.349796 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.350414 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.352446 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.362035 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:14.374749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1396/2789] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:14.378839 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.389742 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2562b39-6dd4-44b6-abe5-e652c2ad9972 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.394362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.394994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.395302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.395574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.396273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.397856 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.407768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1394/2790] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:14.412699 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.422398 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873f62d-c50c-459c-b9a1-7575c8b66506 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.427670 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33139e3-b762-4de4-b7e0-a551cf0fda69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.428275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33139e3-b762-4de4-b7e0-a551cf0fda69 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.428551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33139e3-b762-4de4-b7e0-a551cf0fda69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.428818 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33139e3-b762-4de4-b7e0-a551cf0fda69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.429443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33139e3-b762-4de4-b7e0-a551cf0fda69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.431002 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33139e3-b762-4de4-b7e0-a551cf0fda69 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.441719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1397/2791] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:14.445762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.455659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5bfd9-defd-4081-bab0-2bc62c2a3f48 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.460364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.460997 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.461277 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.461551 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] PATH_INFO: `/v3/projects/f7a3ed6d967a41b4a5fd19f042e0cc37/users/21a3dc0c72d9467d92bb2b1a1399ae3e/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.477019 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a3ed6d967a41b4a5fd19f042e0cc37, user_id=21a3dc0c72d9467d92bb2b1a1399ae3e, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.479801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.492713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1395/2792] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:14 2022] PUT /identity/v3/projects/f7a3ed6d967a41b4a5fd19f042e0cc37/users/21a3dc0c72d9467d92bb2b1a1399ae3e/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:14.502906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.503192 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.503496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.530726 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] MFA Rules not processed for user `21a3dc0c72d9467d92bb2b1a1399ae3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:14.531936 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:14.600423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1398/2793] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:14.604819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.666325 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3abfca-e862-4f17-87f7-981672598c03 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.670975 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.671567 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.671846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.672117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.672768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.674650 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.700397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1396/2794] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:14.704557 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.716919 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb96cfa-2143-4912-a7e1-f9906e563534 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.774538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.775135 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.775408 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.775671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.776338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.778232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.798882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1399/2795] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:14 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:33:14.803047 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.813339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf9d4-cf98-41a0-9f65-222fd00b4e2f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.818260 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.818922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.819202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.819472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.820109 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.821725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.832667 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1397/2796] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:14.836632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.846625 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0e57e-613a-460c-808c-0b919d90a198 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.852018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.852635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.852912 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.853206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] PATH_INFO: `/v3/projects/42d0b4f6d6c0416da4844c6ced24c27d/users/a5e15fac62044aa182d4c32762ab4e9b/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.867780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d0b4f6d6c0416da4844c6ced24c27d, user_id=a5e15fac62044aa182d4c32762ab4e9b, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.870459 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.884762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1400/2797] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:14 2022] PUT /identity/v3/projects/42d0b4f6d6c0416da4844c6ced24c27d/users/a5e15fac62044aa182d4c32762ab4e9b/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:14.888868 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.899541 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07882-763b-49be-98eb-2bbab1b9551f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.953895 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc2f63-4c92-466b-9e37-8283a1aaa5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.954483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2f63-4c92-466b-9e37-8283a1aaa5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.954762 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2f63-4c92-466b-9e37-8283a1aaa5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.955032 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2f63-4c92-466b-9e37-8283a1aaa5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.955660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc2f63-4c92-466b-9e37-8283a1aaa5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.958424 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc2f63-4c92-466b-9e37-8283a1aaa5a3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:14.968579 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1398/2798] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:14.972786 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:14.983044 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6c50-be07-4323-9239-637573e0c3a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:14.987581 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:14.988362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:14.988656 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:14.988929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] PATH_INFO: `/v3/projects/42d0b4f6d6c0416da4844c6ced24c27d/users/a5e15fac62044aa182d4c32762ab4e9b/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:14.990838 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d0b4f6d6c0416da4844c6ced24c27d, user_id=a5e15fac62044aa182d4c32762ab4e9b, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:14.993505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:15.004297 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1401/2799] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:14 2022] PUT /identity/v3/projects/42d0b4f6d6c0416da4844c6ced24c27d/users/a5e15fac62044aa182d4c32762ab4e9b/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:15.015558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:15.016072 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:15.016183 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:15.045771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] MFA Rules not processed for user `a5e15fac62044aa182d4c32762ab4e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:15.046976 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:15.067282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:15.115901 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1399/2800] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:15.126332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:15.128064 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:15.158845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:15.189596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58580519-83bd-47c5-ba29-c97e13de7947 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:15.194033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2be034-dcfa-4a54-995e-10a77ab51fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:15.194635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2be034-dcfa-4a54-995e-10a77ab51fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:15.194913 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2be034-dcfa-4a54-995e-10a77ab51fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:15.195185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2be034-dcfa-4a54-995e-10a77ab51fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:15.204073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2be034-dcfa-4a54-995e-10a77ab51fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:15.205525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2be034-dcfa-4a54-995e-10a77ab51fc0 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:15.214900 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1402/2801] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:16.765268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:16.827030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aaa80-331c-4080-be25-de92ef4cd95a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:16.831695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:16.832859 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:16.833213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:16.833498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] PATH_INFO: `/v3/users/21a3dc0c72d9467d92bb2b1a1399ae3e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:16.834621 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a3dc0c72d9467d92bb2b1a1399ae3e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:16.836580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:16.883827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a3dc0c72d9467d92bb2b1a1399ae3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:16.884494 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a3dc0c72d9467d92bb2b1a1399ae3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:16.885104 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a3dc0c72d9467d92bb2b1a1399ae3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:16.897005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a3dc0c72d9467d92bb2b1a1399ae3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:16.904242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a3dc0c72d9467d92bb2b1a1399ae3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:16.916467 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1400/2802] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:16 2022] DELETE /identity/v3/users/21a3dc0c72d9467d92bb2b1a1399ae3e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:16.920905 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:16.921483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:16.921483 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:16.940453 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:16.941727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:17.000086 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1403/2803] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:17.010406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:17.011815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:17.054215 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:17.071896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1705-1116-4c97-939f-6cb469f848b5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:17.075469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:17.076062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:17.076368 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:17.076636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:17.085352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:17.086760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:17.095912 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1401/2804] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:17.357203 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:17.358719 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:17.372016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4948b3-9a49-4670-8b46-59c38573841c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:17.376547 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:17.377660 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:17.377937 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:17.378390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] PATH_INFO: `/v3/projects/f7a3ed6d967a41b4a5fd19f042e0cc37` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:17.380440 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a3ed6d967a41b4a5fd19f042e0cc37)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:17.383695 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:17.432883 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a3ed6d967a41b4a5fd19f042e0cc37', 'request_id': 'req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.438432 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a3ed6d967a41b4a5fd19f042e0cc37', 'request_id': 'req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.446115 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a3ed6d967a41b4a5fd19f042e0cc37', 'request_id': 'req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.457359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] The token cache is being invalidate because project f7a3ed6d967a41b4a5fd19f042e0cc37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:17.457645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.458299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.459791 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1404/2805] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:17 2022] DELETE /identity/v3/projects/f7a3ed6d967a41b4a5fd19f042e0cc37 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:17.463745 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:17.473923 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e435902-85d8-4b26-9c95-73668260c1d6 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:17.530501 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:17.531120 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:17.531401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:17.531674 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] PATH_INFO: `/v3/users/a5e15fac62044aa182d4c32762ab4e9b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:17.532854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e15fac62044aa182d4c32762ab4e9b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:17.534968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:17.579339 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e15fac62044aa182d4c32762ab4e9b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.579994 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e15fac62044aa182d4c32762ab4e9b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.580636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e15fac62044aa182d4c32762ab4e9b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.593495 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e15fac62044aa182d4c32762ab4e9b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.601783 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f960c6c8-288e-4e20-8cb9-bedcea976217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e15fac62044aa182d4c32762ab4e9b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:17.614304 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1402/2806] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:17 2022] DELETE /identity/v3/users/a5e15fac62044aa182d4c32762ab4e9b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:17.896607 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:17.898042 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:17.911092 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4a4b-18fa-488e-b9f7-d3ba2f6d5dad admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:17.960673 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:17.972010 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:17.972794 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:17.973150 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:17.973429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] PATH_INFO: `/v3/projects/42d0b4f6d6c0416da4844c6ced24c27d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:17.974553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d0b4f6d6c0416da4844c6ced24c27d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:17.976523 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.021647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d0b4f6d6c0416da4844c6ced24c27d', 'request_id': 'req-435b1fa4-eb39-41dc-9118-0b9b22221efb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:18.027335 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d0b4f6d6c0416da4844c6ced24c27d', 'request_id': 'req-435b1fa4-eb39-41dc-9118-0b9b22221efb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:18.035116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d0b4f6d6c0416da4844c6ced24c27d', 'request_id': 'req-435b1fa4-eb39-41dc-9118-0b9b22221efb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:18.046845 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] The token cache is being invalidate because project 42d0b4f6d6c0416da4844c6ced24c27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:18.047069 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:18.047708 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-435b1fa4-eb39-41dc-9118-0b9b22221efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:18.049220 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1405/2807] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:17 2022] DELETE /identity/v3/projects/42d0b4f6d6c0416da4844c6ced24c27d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:18.054592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.055075 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.055175 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.075045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:18.076310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:18.126706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:18.137398 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1403/2808] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:33:18.145611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.145894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.146168 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.165521 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:18.166724 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:18.174572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1406/2809] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:18.178673 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.188409 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fff01f-123e-4981-b9e7-0deffe8fcee9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.194116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.194702 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.194978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.195245 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.195881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:33:18.196178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.197778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.209922 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1404/2810] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:33:18.214296 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.223706 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba581d-0900-4955-ae60-2cdacd4b36ca None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.228388 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.228981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.229258 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.229530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.230153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.232048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.259742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1407/2811] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:18.263916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.273818 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba602ff2-4dd4-482b-9965-11c7898ceb6c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.332029 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.332773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.333054 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.333328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.333954 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.335877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.355722 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1405/2812] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:18.359844 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.370412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298d23e-acbc-403a-a567-d4c6464473fc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.374935 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.375514 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.375788 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.376051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.376711 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.378283 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.389117 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1408/2813] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:18.392988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.402927 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33dde8-825a-4dab-a216-36bd62746418 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.407555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618b48a-1c8b-44b3-836f-4225a0f2e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.408142 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b48a-1c8b-44b3-836f-4225a0f2e89d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.408712 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b48a-1c8b-44b3-836f-4225a0f2e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.408988 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b48a-1c8b-44b3-836f-4225a0f2e89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.409625 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618b48a-1c8b-44b3-836f-4225a0f2e89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.411413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618b48a-1c8b-44b3-836f-4225a0f2e89d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.421916 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1406/2814] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:18.426405 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.436216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970a569-d5c5-4312-b67b-1faa1a278a28 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.440829 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.442061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.442372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.442704 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] PATH_INFO: `/v3/projects/9e92b79691f44b13bc617921e38cf504/users/39176bd517194a1c85b3d34f87ed4c7b/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.457771 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e92b79691f44b13bc617921e38cf504, user_id=39176bd517194a1c85b3d34f87ed4c7b, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.460444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.473872 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1409/2815] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:18 2022] PUT /identity/v3/projects/9e92b79691f44b13bc617921e38cf504/users/39176bd517194a1c85b3d34f87ed4c7b/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:18.541355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.541703 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.541944 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.570285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] MFA Rules not processed for user `39176bd517194a1c85b3d34f87ed4c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:18.571484 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:18.637831 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1407/2816] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:18.642245 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.704093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f670-5caf-4d61-95e0-f5c157ed9d4c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.709519 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.710112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.710390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.710658 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.711267 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.713197 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.740174 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1410/2817] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:18.744741 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.755956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03241695-e5c3-4338-abde-70b899d5c588 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.805326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:18.813098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.813695 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.813970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.814238 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.814842 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.817549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.838126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1408/2818] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:18 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:18.842206 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.847684 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:18.853929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1c2d6-da1e-4af6-85c6-501ecf3cb971 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.859382 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.860004 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.860313 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.860586 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.861215 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.862809 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.873402 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1411/2819] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:18.877584 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.888178 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7783f-a62e-442c-b2a1-43c1c649e569 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.892878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.893518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.893799 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.894071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] PATH_INFO: `/v3/projects/cf50776b47004796a1460e99d10e9bbe/users/b5d95f950b11425f95047fff02d53bf3/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.909405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf50776b47004796a1460e99d10e9bbe, user_id=b5d95f950b11425f95047fff02d53bf3, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.912043 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:18.924796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1409/2820] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:18 2022] PUT /identity/v3/projects/cf50776b47004796a1460e99d10e9bbe/users/b5d95f950b11425f95047fff02d53bf3/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:18.928782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:18.938592 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609a60-e067-4534-ae77-4d97b90797bc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:18.994783 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1a44d-68bf-4893-9a40-805fd68587bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:18.995448 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a44d-68bf-4893-9a40-805fd68587bb admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:18.995672 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a44d-68bf-4893-9a40-805fd68587bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:18.995949 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a44d-68bf-4893-9a40-805fd68587bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:18.996679 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1a44d-68bf-4893-9a40-805fd68587bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:18.998262 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1a44d-68bf-4893-9a40-805fd68587bb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:19.009189 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1412/2821] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:19.013341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:19.023628 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e68b2-e8a8-4f7a-b569-7c08875deb4b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:19.028309 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:19.028967 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:19.029246 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:19.029520 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] PATH_INFO: `/v3/projects/cf50776b47004796a1460e99d10e9bbe/users/b5d95f950b11425f95047fff02d53bf3/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:19.031430 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf50776b47004796a1460e99d10e9bbe, user_id=b5d95f950b11425f95047fff02d53bf3, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:19.034359 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:19.044676 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1410/2822] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:19 2022] PUT /identity/v3/projects/cf50776b47004796a1460e99d10e9bbe/users/b5d95f950b11425f95047fff02d53bf3/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:19.055133 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:19.055417 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:19.055688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:19.083006 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] MFA Rules not processed for user `b5d95f950b11425f95047fff02d53bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:19.084254 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:19.151839 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1413/2823] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:19.163605 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:19.165390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:19.221374 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781334-fcf6-4f3f-8832-d2cc0b9470ce admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:19.225130 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:19.225713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:19.225985 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:19.226249 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:19.234962 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:19.236407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:19.246607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1411/2824] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:19.292797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:19.294198 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:19.306631 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc52107-c4f8-4580-8856-2b042125df18 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:19.310302 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989e870-30c6-4609-8916-fe164c12c18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:19.310892 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989e870-30c6-4609-8916-fe164c12c18a service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:19.311163 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989e870-30c6-4609-8916-fe164c12c18a service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:19.311433 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989e870-30c6-4609-8916-fe164c12c18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:19.368484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989e870-30c6-4609-8916-fe164c12c18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:19.369906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989e870-30c6-4609-8916-fe164c12c18a service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:19.378723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1414/2825] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:22.678092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:22.739657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17008b9-9ddc-4db3-92b6-1205296f2935 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:22.744420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:22.745089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:22.745384 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:22.745641 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] PATH_INFO: `/v3/users/39176bd517194a1c85b3d34f87ed4c7b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:22.746761 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39176bd517194a1c85b3d34f87ed4c7b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:22.748710 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:22.795095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39176bd517194a1c85b3d34f87ed4c7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:22.795754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39176bd517194a1c85b3d34f87ed4c7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:22.796396 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39176bd517194a1c85b3d34f87ed4c7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:22.800928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:22.810729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39176bd517194a1c85b3d34f87ed4c7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:22.817295 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39176bd517194a1c85b3d34f87ed4c7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:22.829222 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1412/2826] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:22 2022] DELETE /identity/v3/users/39176bd517194a1c85b3d34f87ed4c7b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:22.834423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:22.834893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:22.834893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:22.854061 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:22.855240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:22.860095 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:22.913047 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1415/2827] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:22.922635 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:22.924058 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:22.982035 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8eca9a-94a4-4910-9b6d-285f8f11a96f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:22.985691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:22.986294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:22.986570 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:22.986837 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:22.996463 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:22.997900 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:23.008264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1413/2828] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:23.281518 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:23.282917 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:23.296362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dc96-bbe1-44fb-af74-4266e926ec63 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:23.302001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:23.302636 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:23.302909 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:23.303188 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] PATH_INFO: `/v3/projects/9e92b79691f44b13bc617921e38cf504` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:23.304442 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e92b79691f44b13bc617921e38cf504)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:23.306260 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:23.360024 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e92b79691f44b13bc617921e38cf504', 'request_id': 'req-61a17202-f6cc-46f9-bf9b-69045753465c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.366281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e92b79691f44b13bc617921e38cf504', 'request_id': 'req-61a17202-f6cc-46f9-bf9b-69045753465c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.373517 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e92b79691f44b13bc617921e38cf504', 'request_id': 'req-61a17202-f6cc-46f9-bf9b-69045753465c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.385426 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] The token cache is being invalidate because project 9e92b79691f44b13bc617921e38cf504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:23.385768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.386359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.387854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1416/2829] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:23 2022] DELETE /identity/v3/projects/9e92b79691f44b13bc617921e38cf504 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:23.392540 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:23.409353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef556-907f-41b3-a025-c612bdb0ca38 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:23.486644 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:23.487275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:23.487573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:23.487850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] PATH_INFO: `/v3/users/b5d95f950b11425f95047fff02d53bf3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:23.490013 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d95f950b11425f95047fff02d53bf3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:23.492986 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:23.548936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d95f950b11425f95047fff02d53bf3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.549618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d95f950b11425f95047fff02d53bf3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.550213 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d95f950b11425f95047fff02d53bf3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.563082 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d95f950b11425f95047fff02d53bf3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.571020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a3fff307-2586-4263-8baf-a40531289fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d95f950b11425f95047fff02d53bf3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:23.573299 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:23.586259 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1414/2830] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:23 2022] DELETE /identity/v3/users/b5d95f950b11425f95047fff02d53bf3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:23.875160 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:23.876702 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:23.893303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17202-f6cc-46f9-bf9b-69045753465c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:23.954734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:23.955249 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:23.955525 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:23.955797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] PATH_INFO: `/v3/projects/cf50776b47004796a1460e99d10e9bbe` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:23.956971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf50776b47004796a1460e99d10e9bbe)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:23.959025 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:23.997227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:24.008608 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf50776b47004796a1460e99d10e9bbe', 'request_id': 'req-c283fc81-7849-45d2-9230-881524401098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:24.014048 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf50776b47004796a1460e99d10e9bbe', 'request_id': 'req-c283fc81-7849-45d2-9230-881524401098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:24.021588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf50776b47004796a1460e99d10e9bbe', 'request_id': 'req-c283fc81-7849-45d2-9230-881524401098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:24.033369 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] The token cache is being invalidate because project cf50776b47004796a1460e99d10e9bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:24.033922 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:24.034571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c283fc81-7849-45d2-9230-881524401098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:24.036600 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1417/2831] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:23 2022] DELETE /identity/v3/projects/cf50776b47004796a1460e99d10e9bbe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:24.041483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.041908 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.042134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.062185 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:24.063383 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:24.122498 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1415/2832] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:24.130216 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.131020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.131020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.150139 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:24.151340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:24.159566 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1418/2833] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:24.164443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.174296 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31f661-ba24-4d60-9f15-b5feedb1a7cf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.179723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.180385 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.180663 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.180935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.181580 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:33:24.181883 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.183472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.195405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1416/2834] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:33:24.199735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.209613 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470e69-bdbf-4209-a39f-918323ba79b0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.214143 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.214735 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.215011 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.215279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.215884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.217907 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.247327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1419/2835] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:24.252565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.262454 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a51ef-0cff-4a6c-96e2-d0e88de42f79 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.320354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.320955 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.321232 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.321503 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.322134 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.324039 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.346177 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1417/2836] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:24.351484 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.361591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b7497-5549-423b-a9ef-204fb86a8166 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.366481 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.367058 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.367333 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.367599 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.368278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.369816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.380110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1420/2837] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 11:33:24.384011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.394953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0faf0b3-99d4-4ff8-88fb-6d6ad3bfd985 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.399814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e0b85-40b9-4eea-91b4-fad7296e2e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.400452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e0b85-40b9-4eea-91b4-fad7296e2e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.401005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e0b85-40b9-4eea-91b4-fad7296e2e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.401282 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e0b85-40b9-4eea-91b4-fad7296e2e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.401937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e0b85-40b9-4eea-91b4-fad7296e2e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.403521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e0b85-40b9-4eea-91b4-fad7296e2e07 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.416066 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1418/2838] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:24.420363 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.430697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9326bfc-fd4d-4491-b1ca-595ef4f3b1f3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.435345 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.436001 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.436317 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.436588 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] PATH_INFO: `/v3/projects/a06cdd0ab2834943972c0622c9f6187c/users/552f46ae2d8b4fb7bd73c8c47e7cfe71/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.451792 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06cdd0ab2834943972c0622c9f6187c, user_id=552f46ae2d8b4fb7bd73c8c47e7cfe71, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.454462 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.468690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1421/2839] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:24 2022] PUT /identity/v3/projects/a06cdd0ab2834943972c0622c9f6187c/users/552f46ae2d8b4fb7bd73c8c47e7cfe71/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:24.480907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.481236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.481549 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.514011 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] MFA Rules not processed for user `552f46ae2d8b4fb7bd73c8c47e7cfe71`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:24.515243 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:24.561083 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:24.586828 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1419/2840] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:24.591404 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.659236 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ba8b7-2ff9-408b-a85e-61fa2fc9e23c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.664536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.665138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.665418 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.665690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.666307 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.668229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.695021 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1422/2841] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:24.699125 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.709806 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a7cec-fa24-4e31-bae2-4240c7b0439a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.765098 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.765619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.765894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.766161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.766773 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.768688 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.790101 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1420/2842] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:24 2022] POST /identity/v3/users => generated 307 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:33:24.794155 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.805073 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8454ef-f555-4e05-9f63-8f1766b2b3f9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.809727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.810320 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.810603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.810873 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.811504 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.813116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.823665 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1423/2843] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:24.827808 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.837849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8918b3-78d7-46df-a95e-f07d01786f96 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.842390 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.843025 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.843303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.843574 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] PATH_INFO: `/v3/projects/8cb18783d64e49e4943edb3b5e79d8b4/users/cbc8f08d458b4f74958ba92c3a91d5f5/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.858821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb18783d64e49e4943edb3b5e79d8b4, user_id=cbc8f08d458b4f74958ba92c3a91d5f5, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.861496 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.875592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1421/2844] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:24 2022] PUT /identity/v3/projects/8cb18783d64e49e4943edb3b5e79d8b4/users/cbc8f08d458b4f74958ba92c3a91d5f5/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:24.880281 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.890278 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69866396-62cc-47a8-a04a-a346be47ca3d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.922563 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:24.950648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03307f17-fada-4b77-872a-8bc8ab9554c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.951248 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03307f17-fada-4b77-872a-8bc8ab9554c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.951526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03307f17-fada-4b77-872a-8bc8ab9554c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.951795 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03307f17-fada-4b77-872a-8bc8ab9554c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.952465 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03307f17-fada-4b77-872a-8bc8ab9554c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.954050 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03307f17-fada-4b77-872a-8bc8ab9554c1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:24.964295 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1424/2845] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:24.968636 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:24.978937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bcd90-4fcb-4b5c-9bbf-3513a5141995 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:24.983452 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:24.984229 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:24.984510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:24.984776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] PATH_INFO: `/v3/projects/8cb18783d64e49e4943edb3b5e79d8b4/users/cbc8f08d458b4f74958ba92c3a91d5f5/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:24.986655 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb18783d64e49e4943edb3b5e79d8b4, user_id=cbc8f08d458b4f74958ba92c3a91d5f5, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:24.989938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:25.000017 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1422/2846] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:24 2022] PUT /identity/v3/projects/8cb18783d64e49e4943edb3b5e79d8b4/users/cbc8f08d458b4f74958ba92c3a91d5f5/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:25.016775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:25.017098 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:25.017380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:25.053813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] MFA Rules not processed for user `cbc8f08d458b4f74958ba92c3a91d5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:25.055093 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:25.122886 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1425/2847] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:25 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:25.133995 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:25.135844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:25.195268 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f2fd5-60fc-405b-8a24-4e9a9b8655ec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:25.199534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:25.200127 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:25.200435 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:25.200706 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:25.210024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:25.211441 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:25.220890 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1423/2848] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:25.297327 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:25.307376 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70eab5-f90f-4604-bfec-491ce409ab38 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:25.310977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de492-94f3-43ff-ade3-30c6677ec315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:25.311565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de492-94f3-43ff-ade3-30c6677ec315 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:25.311836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de492-94f3-43ff-ade3-30c6677ec315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:25.312101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de492-94f3-43ff-ade3-30c6677ec315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:25.369146 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de492-94f3-43ff-ade3-30c6677ec315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:25.370569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de492-94f3-43ff-ade3-30c6677ec315 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:25.379468 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1426/2849] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:29.608835 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:29.652965 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:29.671823 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1062e-ccb2-46ee-bdd4-a501f9fabe58 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:29.676400 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:29.677064 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:29.677340 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:29.677610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] PATH_INFO: `/v3/users/552f46ae2d8b4fb7bd73c8c47e7cfe71` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:29.678724 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=552f46ae2d8b4fb7bd73c8c47e7cfe71)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:29.680659 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:29.726159 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552f46ae2d8b4fb7bd73c8c47e7cfe71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:29.726734 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552f46ae2d8b4fb7bd73c8c47e7cfe71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:29.727333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552f46ae2d8b4fb7bd73c8c47e7cfe71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:29.738781 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552f46ae2d8b4fb7bd73c8c47e7cfe71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:29.746521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552f46ae2d8b4fb7bd73c8c47e7cfe71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:29.758776 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1424/2850] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:29 2022] DELETE /identity/v3/users/552f46ae2d8b4fb7bd73c8c47e7cfe71 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:29.763474 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:29.763950 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:29.764051 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:29.782979 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:29.784476 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:29.831286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:29.842683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1427/2851] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:33:29.852891 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:29.854275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:29.912397 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108d802-8381-426f-b859-a56f3419ea83 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:29.916095 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:29.916725 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:29.917006 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:29.917276 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:29.926110 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:29.928474 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:29.937788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1425/2852] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:30.200841 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:30.202338 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:30.215507 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa3873-7c0c-400d-9012-a25fb98cd529 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:30.220204 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:30.220918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:30.221225 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:30.221477 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] PATH_INFO: `/v3/projects/a06cdd0ab2834943972c0622c9f6187c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:30.222606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06cdd0ab2834943972c0622c9f6187c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:30.225182 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:30.272571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06cdd0ab2834943972c0622c9f6187c', 'request_id': 'req-f910ceea-7484-4d59-af95-b993227cf76e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.278184 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06cdd0ab2834943972c0622c9f6187c', 'request_id': 'req-f910ceea-7484-4d59-af95-b993227cf76e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.285367 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06cdd0ab2834943972c0622c9f6187c', 'request_id': 'req-f910ceea-7484-4d59-af95-b993227cf76e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.296541 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] The token cache is being invalidate because project a06cdd0ab2834943972c0622c9f6187c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:30.296825 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.297635 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.299111 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1428/2853] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:30 2022] DELETE /identity/v3/projects/a06cdd0ab2834943972c0622c9f6187c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:30.303906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:30.313619 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc71e4-91e0-4fec-9881-a7cb81691a5a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:30.369438 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:30.370062 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:30.370343 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:30.370618 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] PATH_INFO: `/v3/users/cbc8f08d458b4f74958ba92c3a91d5f5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:30.371736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc8f08d458b4f74958ba92c3a91d5f5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:30.373684 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:30.378190 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:30.423403 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc8f08d458b4f74958ba92c3a91d5f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.424030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc8f08d458b4f74958ba92c3a91d5f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.424657 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc8f08d458b4f74958ba92c3a91d5f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.437051 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc8f08d458b4f74958ba92c3a91d5f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.444896 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1c7b278d-7a0c-464e-956d-04ee6d262561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc8f08d458b4f74958ba92c3a91d5f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.456924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1426/2854] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:30 2022] DELETE /identity/v3/users/cbc8f08d458b4f74958ba92c3a91d5f5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:30.727928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:30.729377 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:30.743351 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ceea-7484-4d59-af95-b993227cf76e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:30.800655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:30.801359 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:30.801550 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:30.802467 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] PATH_INFO: `/v3/projects/8cb18783d64e49e4943edb3b5e79d8b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:30.803691 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb18783d64e49e4943edb3b5e79d8b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:30.805572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:30.854770 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb18783d64e49e4943edb3b5e79d8b4', 'request_id': 'req-106f7de1-422d-43b6-894e-a565a67c3ff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.860299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb18783d64e49e4943edb3b5e79d8b4', 'request_id': 'req-106f7de1-422d-43b6-894e-a565a67c3ff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.868618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb18783d64e49e4943edb3b5e79d8b4', 'request_id': 'req-106f7de1-422d-43b6-894e-a565a67c3ff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.879436 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] The token cache is being invalidate because project 8cb18783d64e49e4943edb3b5e79d8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:30.879908 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.880407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-106f7de1-422d-43b6-894e-a565a67c3ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:30.881890 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1429/2855] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:30 2022] DELETE /identity/v3/projects/8cb18783d64e49e4943edb3b5e79d8b4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:33:30.886735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:30.887235 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:30.887353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:30.906711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:30.907924 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:30.967330 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1427/2856] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:30.975659 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:30.975940 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:30.976240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:30.995412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:30.996653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:31.004775 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1430/2857] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 11:33:31.008970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.018666 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a9174-906e-48c4-8faa-93c079b97a9e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.023788 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.024543 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.024827 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.025096 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.025743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:33:31.026038 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.027622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.040092 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1428/2858] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:33:31.044534 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.048955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:31.055831 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c55c-40e0-4ef6-b148-6f81bd6bbafe None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.062747 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.063357 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.063637 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.063910 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.064580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.066955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.101969 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1431/2859] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:31.105049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.114963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc4d2-d021-4377-9a72-24cc991bd2eb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.171236 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.171844 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.172119 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.172421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.173033 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.174947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.191191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:31.196840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1429/2860] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:31.201379 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.213749 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59d122-54aa-4c5f-b28a-468a96131e59 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.218280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.218867 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.219141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.219407 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.220036 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.221653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.232070 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1432/2861] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:31.236107 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.246108 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0734-a9d8-4a95-ab9f-d241a10fb0ce admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.250654 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42063cad-3a8f-4635-893f-f5ba358f213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.251242 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42063cad-3a8f-4635-893f-f5ba358f213e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.251550 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42063cad-3a8f-4635-893f-f5ba358f213e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.251819 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42063cad-3a8f-4635-893f-f5ba358f213e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.252488 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42063cad-3a8f-4635-893f-f5ba358f213e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.254067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42063cad-3a8f-4635-893f-f5ba358f213e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.264521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1430/2862] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:31.268488 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.279378 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46acdeb-874c-4cc1-867d-a3b0ebc57853 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.284179 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.284843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.285116 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.285380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] PATH_INFO: `/v3/projects/7be106c6e2404583ace1d747b30c603f/users/5657de646afc4ddf936f1517d5ad9164/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.299781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be106c6e2404583ace1d747b30c603f, user_id=5657de646afc4ddf936f1517d5ad9164, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.303027 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.315685 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1433/2863] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:31 2022] PUT /identity/v3/projects/7be106c6e2404583ace1d747b30c603f/users/5657de646afc4ddf936f1517d5ad9164/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:31.327748 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e352d-200c-409c-9538-d3489e143442 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.328031 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e352d-200c-409c-9538-d3489e143442 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.328341 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e352d-200c-409c-9538-d3489e143442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.355989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a95e352d-200c-409c-9538-d3489e143442 None None] MFA Rules not processed for user `5657de646afc4ddf936f1517d5ad9164`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:31.357250 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a95e352d-200c-409c-9538-d3489e143442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:31.420606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1431/2864] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:31.425072 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.486482 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61b1e-ee56-4f94-8ceb-0429c97ed237 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.491737 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.492362 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.492644 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.492915 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.493530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.495430 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.521623 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1434/2865] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:31.525777 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e352d-200c-409c-9538-d3489e143442 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.537138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e352d-200c-409c-9538-d3489e143442 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.593894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.594418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.594699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.594968 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.595578 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.597514 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.618899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1432/2866] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:31.623122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.633390 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89731-6863-4230-99cf-ccc5e7568aaa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.637998 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.638587 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.638863 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.639131 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.639760 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.641386 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.651683 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1435/2867] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:31.655566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.666402 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38027db-27e5-4d60-b28d-ca25ac7d7739 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.671012 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec1889-cd39-4515-a90f-512ae64c2cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.671595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec1889-cd39-4515-a90f-512ae64c2cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.671870 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec1889-cd39-4515-a90f-512ae64c2cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.672136 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec1889-cd39-4515-a90f-512ae64c2cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.672847 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec1889-cd39-4515-a90f-512ae64c2cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.674418 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec1889-cd39-4515-a90f-512ae64c2cfa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.684935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1433/2868] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:31.688884 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.698688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1e717-f2cb-4377-9745-11eaf03fe214 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.703229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.703930 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.704229 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.704506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] PATH_INFO: `/v3/projects/97110613c3c54831875214b5f243ec64/users/c18780882f75484598525889cb4eab66/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.719549 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97110613c3c54831875214b5f243ec64, user_id=c18780882f75484598525889cb4eab66, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.722208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.735205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1436/2869] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:31 2022] PUT /identity/v3/projects/97110613c3c54831875214b5f243ec64/users/c18780882f75484598525889cb4eab66/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:33:31.745740 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.746048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.746323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.773953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] MFA Rules not processed for user `c18780882f75484598525889cb4eab66`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:31.775152 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:31.820680 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:31.841805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1434/2870] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:31.846385 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.902837 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:31.909743 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a76e8-3e07-4410-b453-1f2efc31c255 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:31.915739 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:31.916374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:31.916655 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:31.916955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:31.917565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:31.919512 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:31.946075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1437/2871] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:31.950355 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:31.961573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9317a3-e1eb-4f4f-bcc1-75358a9c1a49 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:32.017825 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:32.018420 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.018700 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.018974 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.019595 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:32.022149 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:32.043894 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1435/2872] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:31 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:32.047893 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:32.058162 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1494572-efab-4f92-b0f5-40c94276f387 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:32.062987 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:32.063567 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.063918 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.064205 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.064843 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:32.066423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:32.077063 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1438/2873] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:32.081071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:32.092049 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babecdc7-e066-43f0-a376-035824eb2f12 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:32.096768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:32.097353 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.097631 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.097902 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] PATH_INFO: `/v3/projects/73f14b34494546ec890a494642bda9e4/users/e4e392742db54e10852f1e2aa6422861/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.112914 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f14b34494546ec890a494642bda9e4, user_id=e4e392742db54e10852f1e2aa6422861, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:32.115957 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:32.128810 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1436/2874] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:32 2022] PUT /identity/v3/projects/73f14b34494546ec890a494642bda9e4/users/e4e392742db54e10852f1e2aa6422861/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:32.133349 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:32.143273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cab71a-fb5d-4795-9e7b-c36a8e244de3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:32.199734 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a596df5-4153-4289-ac3c-633574f1b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:32.200373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596df5-4153-4289-ac3c-633574f1b0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.200727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596df5-4153-4289-ac3c-633574f1b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.201013 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596df5-4153-4289-ac3c-633574f1b0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.201650 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a596df5-4153-4289-ac3c-633574f1b0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:32.203242 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a596df5-4153-4289-ac3c-633574f1b0a3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:32.212981 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1439/2875] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:32.217135 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:32.227983 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85cba-6636-4450-b507-287c49760e7b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:32.232699 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:32.233344 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.233623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.233899 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] PATH_INFO: `/v3/projects/73f14b34494546ec890a494642bda9e4/users/e4e392742db54e10852f1e2aa6422861/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.235821 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f14b34494546ec890a494642bda9e4, user_id=e4e392742db54e10852f1e2aa6422861, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:32.238480 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:32.249485 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1437/2876] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:32 2022] PUT /identity/v3/projects/73f14b34494546ec890a494642bda9e4/users/e4e392742db54e10852f1e2aa6422861/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:32.260645 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.260932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.261208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.289332 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] MFA Rules not processed for user `e4e392742db54e10852f1e2aa6422861`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:32.290526 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:32.359126 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1440/2877] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:32.370522 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:32.372303 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:32.431073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3118106-cc71-4334-b192-f638698cf80d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:32.434767 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:32.435364 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:32.435643 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:32.435917 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:32.488079 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:32.492877 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:32.494328 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:32.503606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1438/2878] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:34.005663 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:34.007200 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:34.019017 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c006312-5bd3-4ba4-87fa-2c566879d412 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:34.022628 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:34.023235 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:34.023495 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:34.023764 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:34.071727 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:34.082664 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:34.084697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:34.093506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1441/2879] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:34.615606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:34.624472 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94effe24-612d-44c1-b299-d169281659f2 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:34.628094 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300db804-0658-4eb5-886c-633f1860150f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefa27e6907c4ba28483d31a68753c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:34.628730 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300db804-0658-4eb5-886c-633f1860150f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:34.629005 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300db804-0658-4eb5-886c-633f1860150f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:34.629274 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300db804-0658-4eb5-886c-633f1860150f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:34.638836 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300db804-0658-4eb5-886c-633f1860150f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:34.640262 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300db804-0658-4eb5-886c-633f1860150f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:34.650254 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1439/2880] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:38.531540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:38.597420 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0139403-9c46-4d7e-a60b-40f7dd599965 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:38.605780 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:38.606409 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:38.606688 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:38.606984 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] PATH_INFO: `/v3/users/5657de646afc4ddf936f1517d5ad9164` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:38.608108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5657de646afc4ddf936f1517d5ad9164)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:38.610094 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:38.655177 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5657de646afc4ddf936f1517d5ad9164'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:38.668141 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5657de646afc4ddf936f1517d5ad9164'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:38.675805 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5657de646afc4ddf936f1517d5ad9164'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:38.676475 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5657de646afc4ddf936f1517d5ad9164'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:38.677294 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5657de646afc4ddf936f1517d5ad9164'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:38.689352 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1442/2881] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:38 2022] DELETE /identity/v3/users/5657de646afc4ddf936f1517d5ad9164 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:38.694099 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:38.694607 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:38.694729 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:38.713805 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:38.714943 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:38.774850 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1440/2882] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:38.784491 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:38.785972 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:38.843707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40567c-b531-407e-b96e-6e0b2a3d47b0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:38.847384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:38.847977 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:38.848287 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:38.848596 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:38.857924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:38.859340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:38.869062 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1443/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:39.139566 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:39.141286 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:39.154194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acfed1-4a9c-4304-87c3-732a2bcca954 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:39.158989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:39.159616 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:39.159887 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:39.160155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] PATH_INFO: `/v3/projects/7be106c6e2404583ace1d747b30c603f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:39.161592 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be106c6e2404583ace1d747b30c603f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:39.165056 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:39.213255 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be106c6e2404583ace1d747b30c603f', 'request_id': 'req-86c8653b-2f4e-4057-a2d2-aff913630220'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.220956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be106c6e2404583ace1d747b30c603f', 'request_id': 'req-86c8653b-2f4e-4057-a2d2-aff913630220'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.227115 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be106c6e2404583ace1d747b30c603f', 'request_id': 'req-86c8653b-2f4e-4057-a2d2-aff913630220'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.237182 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] The token cache is being invalidate because project 7be106c6e2404583ace1d747b30c603f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:39.237465 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.238105 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.240285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1441/2884] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:39 2022] DELETE /identity/v3/projects/7be106c6e2404583ace1d747b30c603f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:39.244266 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:39.254020 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e736-97e0-41d8-b8f3-45b828252cc1 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:39.262241 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:39.313971 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:39.314605 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:39.314882 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:39.315153 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] PATH_INFO: `/v3/users/c18780882f75484598525889cb4eab66` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:39.316311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18780882f75484598525889cb4eab66)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:39.318443 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:39.364540 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18780882f75484598525889cb4eab66'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.376102 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18780882f75484598525889cb4eab66'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.383698 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18780882f75484598525889cb4eab66'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.384356 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18780882f75484598525889cb4eab66'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.384965 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18780882f75484598525889cb4eab66'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.397713 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1444/2885] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:39 2022] DELETE /identity/v3/users/c18780882f75484598525889cb4eab66 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:39.670471 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:39.671936 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:39.686612 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8653b-2f4e-4057-a2d2-aff913630220 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:39.745752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:39.746499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:39.746623 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:39.746888 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] PATH_INFO: `/v3/projects/97110613c3c54831875214b5f243ec64` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:39.748004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97110613c3c54831875214b5f243ec64)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:39.749933 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:39.756670 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:39.797593 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97110613c3c54831875214b5f243ec64', 'request_id': 'req-353fd03f-d1d0-4f28-98c5-8445863fb159'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.804889 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97110613c3c54831875214b5f243ec64', 'request_id': 'req-353fd03f-d1d0-4f28-98c5-8445863fb159'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.811126 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97110613c3c54831875214b5f243ec64', 'request_id': 'req-353fd03f-d1d0-4f28-98c5-8445863fb159'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.821815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] The token cache is being invalidate because project 97110613c3c54831875214b5f243ec64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:39.822102 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.822735 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.824199 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1442/2886] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:39 2022] DELETE /identity/v3/projects/97110613c3c54831875214b5f243ec64 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:39.828286 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:39.829772 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:39.842999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e141-c2b2-4228-9a79-d2e5c11832d7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:39.898960 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:39.899578 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:39.899858 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:39.900127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] PATH_INFO: `/v3/users/e4e392742db54e10852f1e2aa6422861` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:39.901279 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e392742db54e10852f1e2aa6422861)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:39.903185 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:39.935496 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:39.949271 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e392742db54e10852f1e2aa6422861'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.961798 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e392742db54e10852f1e2aa6422861'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.969012 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e392742db54e10852f1e2aa6422861'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.969617 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e392742db54e10852f1e2aa6422861'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.970274 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-013599d3-2154-4263-8e76-c89687a831a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e392742db54e10852f1e2aa6422861'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:39.982537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1445/2887] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:33:39 2022] DELETE /identity/v3/users/e4e392742db54e10852f1e2aa6422861 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:40.261483 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.262978 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:40.276333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fd03f-d1d0-4f28-98c5-8445863fb159 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.335161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.335744 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.336021 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.336329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] PATH_INFO: `/v3/projects/73f14b34494546ec890a494642bda9e4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.337434 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f14b34494546ec890a494642bda9e4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.339305 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.386173 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f14b34494546ec890a494642bda9e4', 'request_id': 'req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:40.393524 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f14b34494546ec890a494642bda9e4', 'request_id': 'req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:40.399846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f14b34494546ec890a494642bda9e4', 'request_id': 'req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:40.411878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] The token cache is being invalidate because project 73f14b34494546ec890a494642bda9e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:33:40.412294 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:40.412945 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-921bccd5-c9f8-4e78-bf6c-22fb9fed3b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:33:40.414450 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1443/2888] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:33:40 2022] DELETE /identity/v3/projects/73f14b34494546ec890a494642bda9e4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:40.420387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.420774 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.421056 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.446201 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:40.447431 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:40.509290 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1446/2889] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:40.517133 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.517419 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.517711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.537677 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:40.538878 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:40.546727 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1444/2890] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:40.551348 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.561374 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a771168-a82c-49d4-b1ff-4e782ef9fbb4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.566037 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.566632 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.566906 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.567217 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.567988 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 11:33:40.568316 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.569899 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.581214 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1447/2891] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Aug 18 11:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 11:33:40.585768 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.595165 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63994ef1-9c1a-4d86-9517-e2e78fbf6920 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.600526 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.601123 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.601401 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.601698 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.602323 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.604214 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.621354 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:40.635073 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1445/2892] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:40 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:40.639551 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.650033 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331574ee-4d1c-48d4-a52d-b80ff3fba523 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.705634 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.706227 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.706505 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.706776 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.707384 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.709948 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.733569 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1448/2893] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:40 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 11:33:40.737906 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.748314 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74131f45-46dd-46cf-b1ee-1bed6f53361b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.752928 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.753518 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.753796 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.754068 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.754778 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.756407 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.766155 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1446/2894] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:40.770085 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.780449 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43047f2f-6829-4e83-a9bd-a64a14f800a4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.785368 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64ff7d-6e4c-4edc-a620-195603565e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.786575 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64ff7d-6e4c-4edc-a620-195603565e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.786854 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64ff7d-6e4c-4edc-a620-195603565e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.787124 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64ff7d-6e4c-4edc-a620-195603565e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.787753 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64ff7d-6e4c-4edc-a620-195603565e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.789392 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64ff7d-6e4c-4edc-a620-195603565e50 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.799820 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1449/2895] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:40.803949 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.813754 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251fdbc-5c7b-4711-81ef-27253c29c01e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:40.818642 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:40.819273 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.819548 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.819816 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] PATH_INFO: `/v3/projects/f215c3a85f1c428496d1862a5c72067c/users/2b45c3883c61400aa38fc950ecb49eee/roles/866bc5028f7b4637aa1ae91761cd03b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.836138 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f215c3a85f1c428496d1862a5c72067c, user_id=2b45c3883c61400aa38fc950ecb49eee, role_id=866bc5028f7b4637aa1ae91761cd03b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:40.839280 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:40.852873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1447/2896] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:40 2022] PUT /identity/v3/projects/f215c3a85f1c428496d1862a5c72067c/users/2b45c3883c61400aa38fc950ecb49eee/roles/866bc5028f7b4637aa1ae91761cd03b5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:40.857916 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ba4c1-afab-4ad6-b0a4-39984ea23a15 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:40.858415 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ba4c1-afab-4ad6-b0a4-39984ea23a15 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:40.858548 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ba4c1-afab-4ad6-b0a4-39984ea23a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:40.878928 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a93ba4c1-afab-4ad6-b0a4-39984ea23a15 None None] MFA Rules not processed for user `f292716c40bd49538840171d62e79f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:40.880222 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a93ba4c1-afab-4ad6-b0a4-39984ea23a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:40.890045 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:40.943647 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1450/2897] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:40.953645 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:40.955352 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:41.015801 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f351c8-8228-4287-a814-c641b90bc33b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:41.020860 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:41.021510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:41.022337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:41.022337 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:41.032089 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:41.033555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:41.043963 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1448/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:45.051573 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.052110 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.052591 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.085787 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] MFA Rules not processed for user `2b45c3883c61400aa38fc950ecb49eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:45.087019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:45.158715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1451/2899] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:45.163947 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:45.175854 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45efa4f-d217-497d-84ab-f74db074c67f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:45.182244 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:45.183067 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.183537 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.183814 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.184499 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:45.186372 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:45.219316 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1449/2900] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Aug 18 11:33:45 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:45.224127 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:45.238999 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6199-d5d6-403f-adaf-8b4923b6cfa1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:45.309955 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:45.310782 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.311022 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.311298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.312410 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:45.313914 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:45.338323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1452/2901] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Aug 18 11:33:45 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 11:33:45.342264 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:45.354470 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bc438-5578-43e0-9f15-985e82cee325 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:45.359658 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:45.360326 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.360610 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.360938 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.362257 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:45.364561 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:45.376739 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1450/2902] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:45.381311 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:45.393240 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88014759-1c4d-4ce7-bc7d-bc9583cad14a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:45.398671 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:45.399423 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.399768 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.400112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] PATH_INFO: `/v3/projects/c3c85d5165774e80812175604ae1d2cf/users/abd130e70dad48219834b8cc3aee107d/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.419326 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c85d5165774e80812175604ae1d2cf, user_id=abd130e70dad48219834b8cc3aee107d, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:45.422299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:45.438310 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1453/2903] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:45 2022] PUT /identity/v3/projects/c3c85d5165774e80812175604ae1d2cf/users/abd130e70dad48219834b8cc3aee107d/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 11:33:45.443907 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:45.454972 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ebbf5-acd6-4178-b744-4ca3f21cc8a2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:45.532513 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e040b79-6da7-41d6-b22d-58c9ad17fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:45.533204 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040b79-6da7-41d6-b22d-58c9ad17fb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.533565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040b79-6da7-41d6-b22d-58c9ad17fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.534413 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040b79-6da7-41d6-b22d-58c9ad17fb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.535362 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e040b79-6da7-41d6-b22d-58c9ad17fb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:45.537230 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e040b79-6da7-41d6-b22d-58c9ad17fb1a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:45.548743 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1451/2904] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Aug 18 11:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 11:33:45.553500 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:45.564372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41101a3b-5de7-42bd-a9b5-ea16ca2d1200 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:45.570138 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:45.570797 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:45.571075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:45.571344 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] PATH_INFO: `/v3/projects/c3c85d5165774e80812175604ae1d2cf/users/abd130e70dad48219834b8cc3aee107d/roles/cd03b4f182254618b6e0a071d21dd304` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:45.573299 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c85d5165774e80812175604ae1d2cf, user_id=abd130e70dad48219834b8cc3aee107d, role_id=cd03b4f182254618b6e0a071d21dd304)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:45.575975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:45.582323 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:45.589887 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1454/2905] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Aug 18 11:33:45 2022] PUT /identity/v3/projects/c3c85d5165774e80812175604ae1d2cf/users/abd130e70dad48219834b8cc3aee107d/roles/cd03b4f182254618b6e0a071d21dd304 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:33:50.205310 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef2e5-0c66-47f8-a24c-61c7ff57a2b4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:50.205849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef2e5-0c66-47f8-a24c-61c7ff57a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:50.205849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef2e5-0c66-47f8-a24c-61c7ff57a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:50.234840 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-39eef2e5-0c66-47f8-a24c-61c7ff57a2b4 None None] MFA Rules not processed for user `abd130e70dad48219834b8cc3aee107d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:50.236046 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-39eef2e5-0c66-47f8-a24c-61c7ff57a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:50.249873 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:33:50.318297 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1452/2906] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:33:50.334717 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:33:50.336499 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:50.413742 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f7db-d1e0-498c-9f2d-75376760d337 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:33:50.419530 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:33:50.420147 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:50.420461 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:50.420732 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:50.480932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:33:50.482618 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:33:50.496929 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1455/2907] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:33:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:33:51.780024 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:33:51.780572 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:33:51.780691 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:33:51.800285 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] MFA Rules not processed for user `2b45c3883c61400aa38fc950ecb49eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 11:33:51.801705 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:33:51.809416 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1453/2908] 10.222.0.16 () {58 vars in 958 bytes} [Thu Aug 18 11:33:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 11:34:00.488075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=57 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:34:01.794881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:34:15.589944 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:34:15.591339 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:34:15.729715 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d9b3b-2162-4d82-aeb4-24e5d1b12090 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:34:15.734836 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:34:15.735439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:34:15.735716 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:34:15.735986 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:34:15.737816 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:34:15.749340 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:34:15.750902 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:34:15.761611 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1456/2909] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:34:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:34:25.756529 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=63 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:34:33.016660 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:34:33.018128 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:34:33.108521 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df0b35-4cde-4547-a230-47c41b897f36 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:34:33.112114 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:34:33.112736 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:34:33.113048 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:34:33.113315 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:34:33.122760 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:34:33.124162 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:34:33.133468 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1454/2910] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:34:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:34:43.129202 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=41 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:36:53.060889 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:36:53.070300 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37977ff-4858-464b-8ba1-ad9988955870 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:36:53.074781 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:36:53.075372 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:36:53.075653 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:36:53.075924 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:36:53.077690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:36:53.112112 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:36:53.113562 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:36:53.135387 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1457/2911] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:36:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:36:54.323942 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:36:54.332573 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18009295-d01a-4199-8105-90fc2bda9f17 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:36:54.336153 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:36:54.336771 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:36:54.337144 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:36:54.337615 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:36:54.339665 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:36:54.353405 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:36:54.354817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:36:54.366302 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1455/2912] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:36:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:03.131983 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=18 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:37:04.361141 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:37:06.264259 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:37:06.272813 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037579e9-27f3-4186-b1ef-3222ef23b233 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:37:06.276439 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:37:06.277028 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:37:06.277303 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:37:06.277571 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:37:06.279280 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:06.288975 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:37:06.290328 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:37:06.298346 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1458/2913] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:06.409713 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:37:06.419139 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8b219-9850-4570-8616-45e95d37a078 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:37:06.422757 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:37:06.423346 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:37:06.423620 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:37:06.423886 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:37:06.425606 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:06.436086 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:37:06.437622 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:37:06.445711 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1456/2914] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:06.559053 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:37:06.567652 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa94e3-5704-4d6a-99e1-97c216666b62 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:37:06.571252 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:37:06.571846 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:37:06.572122 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:37:06.572429 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:37:06.574134 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:06.587572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:37:06.589074 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:37:06.600091 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1459/2915] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:37:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:06.663026 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:37:06.664443 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:06.679207 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdb893-409c-4dec-8116-28df0103e2db service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:37:06.684466 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:37:06.685045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:37:06.685322 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:37:06.685586 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:37:06.687197 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:06.698921 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:37:06.700510 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:37:06.710333 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1457/2916] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:37:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:08.981894 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:37:08.983538 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:09.000130 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067714a8-14b4-4a19-8e6e-8d446652e94d service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:37:09.005824 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:37:09.006373 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:37:09.006654 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:37:09.006932 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:37:09.008676 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:09.022108 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:37:09.023690 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:37:09.035874 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1460/2917] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:37:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:15.390991 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:37:15.401194 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a91b49-b3af-4572-8916-12762a997dcf service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:37:15.404815 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:37:15.405406 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:37:15.405685 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:37:15.405956 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:37:15.407715 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:37:15.417752 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:37:15.419158 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:37:15.427191 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1458/2918] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:37:19.031232 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=18 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:37:25.423469 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=18 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:38:54.745052 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:38:54.754506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac6c2-d0e9-4d38-86e9-a5307339c48d service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:38:54.758031 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:38:54.758606 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:38:54.758879 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:38:54.759145 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:38:54.767648 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:38:54.769084 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:38:54.777536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1461/2919] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:38:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:38:55.643853 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:38:55.660421 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f197f5-4aa9-46b7-b121-e10ec0dfe821 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:38:55.666689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f43ff51c5f84d34978c46bce6db67ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:38:55.667329 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:38:55.667609 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:38:55.667881 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:38:55.670170 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:38:55.683020 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:38:55.684629 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:38:55.695849 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1459/2920] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:38:59.489759 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:38:59.499261 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387fa0a-bbe3-4d12-98fd-11d0dcae3072 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:38:59.503535 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09c08eca5be4a1b8977ff91bb468358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:38:59.504464 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:38:59.504745 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:38:59.505016 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:38:59.574289 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:38:59.575657 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:38:59.596019 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1462/2921] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:39:02.708817 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:39:02.736266 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfa985-2771-4ff5-b7c9-e3a575f52723 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:39:02.741723 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:39:02.742690 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:39:02.743076 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:39:02.743444 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:39:02.747007 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:39:02.767116 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:39:02.769004 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:39:02.787203 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1460/2922] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:39:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:39:05.954071 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:39:05.968603 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41574c-c2bd-45d2-be14-d8644286efd5 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:39:05.973366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:39:05.974009 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:39:05.974273 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:39:05.974553 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] PATH_INFO: `/v3/users/2b45c3883c61400aa38fc950ecb49eee` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:39:05.975677 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b45c3883c61400aa38fc950ecb49eee)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:39:05.977627 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:39:06.031572 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b45c3883c61400aa38fc950ecb49eee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.043492 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b45c3883c61400aa38fc950ecb49eee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.051565 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b45c3883c61400aa38fc950ecb49eee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.052366 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b45c3883c61400aa38fc950ecb49eee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.053088 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b45c3883c61400aa38fc950ecb49eee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.066075 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1463/2923] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:39:05 2022] DELETE /identity/v3/users/2b45c3883c61400aa38fc950ecb49eee => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:39:06.236016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:39:06.246864 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7532285-a1e8-4a8c-9702-ad73d18ab372 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:39:06.303334 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:39:06.303882 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:39:06.304161 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:39:06.304461 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] PATH_INFO: `/v3/projects/f215c3a85f1c428496d1862a5c72067c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:39:06.305565 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f215c3a85f1c428496d1862a5c72067c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:39:06.307456 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:39:06.353370 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f215c3a85f1c428496d1862a5c72067c', 'request_id': 'req-b1be7d84-9061-4924-8224-152f4528ef74'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.360661 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f215c3a85f1c428496d1862a5c72067c', 'request_id': 'req-b1be7d84-9061-4924-8224-152f4528ef74'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.367154 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f215c3a85f1c428496d1862a5c72067c', 'request_id': 'req-b1be7d84-9061-4924-8224-152f4528ef74'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.390030 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] The token cache is being invalidate because project f215c3a85f1c428496d1862a5c72067c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:39:06.390312 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.390937 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.392442 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1461/2924] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:39:06 2022] DELETE /identity/v3/projects/f215c3a85f1c428496d1862a5c72067c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:39:06.396444 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:39:06.397823 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:39:06.411018 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce013e2-84eb-4011-abf7-5465e36d402d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:39:06.444707 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:39:06.471639 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:39:06.472298 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:39:06.472580 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:39:06.472851 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] PATH_INFO: `/v3/users/abd130e70dad48219834b8cc3aee107d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:39:06.473973 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd130e70dad48219834b8cc3aee107d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:39:06.475868 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:39:06.522762 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd130e70dad48219834b8cc3aee107d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.534945 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd130e70dad48219834b8cc3aee107d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.542486 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd130e70dad48219834b8cc3aee107d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.543101 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd130e70dad48219834b8cc3aee107d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.543697 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd130e70dad48219834b8cc3aee107d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.556380 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1464/2925] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Aug 18 11:39:06 2022] DELETE /identity/v3/users/abd130e70dad48219834b8cc3aee107d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:39:06.729335 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:39:06.730755 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:39:06.744534 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be7d84-9061-4924-8224-152f4528ef74 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:39:06.793951 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:39:06.804358 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f292716c40bd49538840171d62e79f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8a7d63a241745a88c5287cb0d12f0c9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:39:06.805275 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:39:06.805558 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:39:06.805846 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] PATH_INFO: `/v3/projects/c3c85d5165774e80812175604ae1d2cf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:39:06.807071 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c85d5165774e80812175604ae1d2cf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:39:06.808970 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:39:06.858045 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c85d5165774e80812175604ae1d2cf', 'request_id': 'req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.864953 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c85d5165774e80812175604ae1d2cf', 'request_id': 'req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.871117 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c85d5165774e80812175604ae1d2cf', 'request_id': 'req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.881935 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] The token cache is being invalidate because project c3c85d5165774e80812175604ae1d2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 11:39:06.882224 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.882865 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 11:39:06.884373 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1462/2926] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Aug 18 11:39:06 2022] DELETE /identity/v3/projects/c3c85d5165774e80812175604ae1d2cf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 11:39:16.552806 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:39:16.878989 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:39:23.231275 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:39:23.232723 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:39:23.301322 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77c917-e8fc-4a50-8705-c85f8680f0b9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:39:23.358980 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866043dc-3f56-4974-8185-004070f11046 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:39:23.360325 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866043dc-3f56-4974-8185-004070f11046 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:39:23.360506 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866043dc-3f56-4974-8185-004070f11046 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:39:23.360777 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866043dc-3f56-4974-8185-004070f11046 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:39:23.362537 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-866043dc-3f56-4974-8185-004070f11046 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:39:23.373536 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866043dc-3f56-4974-8185-004070f11046 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:39:23.375675 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866043dc-3f56-4974-8185-004070f11046 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:39:23.385412 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1465/2927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:39:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:39:33.381208 dv-cinder-853450-2-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=58 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 11:44:28.190604 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 11:44:28.192016 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 11:44:28.264555 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb6f4c-b4f6-4c9c-a75b-16cda6a701fb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 11:44:28.270081 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488c62f-71b3-4b51-ae40-b1a051e2067e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f669319338ab4c6ea22dead596391789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bdd785798ec4a6ba0c5ebd91d110098', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 11:44:28.270678 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488c62f-71b3-4b51-ae40-b1a051e2067e service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 11:44:28.270959 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488c62f-71b3-4b51-ae40-b1a051e2067e service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 11:44:28.271233 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488c62f-71b3-4b51-ae40-b1a051e2067e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 11:44:28.279689 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488c62f-71b3-4b51-ae40-b1a051e2067e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 11:44:28.281313 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488c62f-71b3-4b51-ae40-b1a051e2067e service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 11:44:28.291885 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1463/2928] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Aug 18 11:44:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 11:44:38.287497 dv-cinder-853450-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=31 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}